May 21 11:28:01.321261 ip-10-0-139-213 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory May 21 11:28:01.321273 ip-10-0-139-213 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory May 21 11:28:01.321280 ip-10-0-139-213 systemd[1]: kubelet.service: Failed with result 'resources'. May 21 11:28:01.321560 ip-10-0-139-213 systemd[1]: Failed to start Kubernetes Kubelet. May 21 11:28:11.534470 ip-10-0-139-213 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. May 21 11:28:11.534492 ip-10-0-139-213 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 686fa09a004a42cdbf0ab495e65f6715 -- May 21 11:30:44.987366 ip-10-0-139-213 systemd[1]: Starting Kubernetes Kubelet... May 21 11:30:45.418848 ip-10-0-139-213 kubenswrapper[2564]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. May 21 11:30:45.418848 ip-10-0-139-213 kubenswrapper[2564]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. May 21 11:30:45.418848 ip-10-0-139-213 kubenswrapper[2564]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. May 21 11:30:45.418848 ip-10-0-139-213 kubenswrapper[2564]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. May 21 11:30:45.418848 ip-10-0-139-213 kubenswrapper[2564]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. May 21 11:30:45.421195 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.421107 2564 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" May 21 11:30:45.426617 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426601 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 21 11:30:45.426656 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426618 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 21 11:30:45.426656 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426623 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 21 11:30:45.426656 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426626 2564 feature_gate.go:328] unrecognized feature gate: NewOLM May 21 11:30:45.426656 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426629 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages May 21 11:30:45.426656 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426632 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 21 11:30:45.426656 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426635 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 21 11:30:45.426656 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426637 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 21 11:30:45.426656 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426641 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 21 11:30:45.426656 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426644 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 21 11:30:45.426656 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426647 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 21 11:30:45.426656 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426649 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 21 11:30:45.426656 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426652 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 21 11:30:45.426656 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426654 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 21 11:30:45.426656 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426658 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 21 11:30:45.426656 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426662 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 21 11:30:45.426656 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426664 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 21 11:30:45.427031 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426668 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 21 11:30:45.427031 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426670 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 21 11:30:45.427031 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426673 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 21 11:30:45.427031 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426675 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 21 11:30:45.427031 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426678 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 21 11:30:45.427031 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426680 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 21 11:30:45.427031 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426683 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 21 11:30:45.427031 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426685 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 21 11:30:45.427031 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426688 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 21 11:30:45.427031 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426690 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 21 11:30:45.427031 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426693 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 21 11:30:45.427031 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426695 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability May 21 11:30:45.427031 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426699 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 21 11:30:45.427031 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426701 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 21 11:30:45.427031 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426704 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 21 11:30:45.427031 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426706 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 21 11:30:45.427031 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426708 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 21 11:30:45.427031 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426711 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 21 11:30:45.427031 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426713 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 21 11:30:45.427474 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426716 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 21 11:30:45.427474 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426718 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 21 11:30:45.427474 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426720 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 21 11:30:45.427474 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426723 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 21 11:30:45.427474 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426725 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 21 11:30:45.427474 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426727 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 21 11:30:45.427474 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426730 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 21 11:30:45.427474 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426733 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 21 11:30:45.427474 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426735 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 21 11:30:45.427474 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426738 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 21 11:30:45.427474 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426740 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 21 11:30:45.427474 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426743 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 21 11:30:45.427474 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426746 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 21 11:30:45.427474 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426748 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 21 11:30:45.427474 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426755 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 21 11:30:45.427474 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426760 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 21 11:30:45.427474 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426763 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 21 11:30:45.427474 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426766 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 21 11:30:45.427474 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426769 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 21 11:30:45.427950 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426774 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 21 11:30:45.427950 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426779 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 21 11:30:45.427950 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426781 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 21 11:30:45.427950 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426784 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 21 11:30:45.427950 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426786 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 21 11:30:45.427950 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426789 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 21 11:30:45.427950 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426792 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores May 21 11:30:45.427950 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426795 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 21 11:30:45.427950 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426797 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 21 11:30:45.427950 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426799 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 21 11:30:45.427950 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426802 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 21 11:30:45.427950 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426806 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 21 11:30:45.427950 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426809 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 21 11:30:45.427950 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426811 2564 feature_gate.go:328] unrecognized feature gate: Example May 21 11:30:45.427950 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426814 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 21 11:30:45.427950 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426816 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 21 11:30:45.427950 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426818 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 21 11:30:45.427950 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426821 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 21 11:30:45.427950 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426823 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 21 11:30:45.427950 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426826 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 21 11:30:45.428436 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426828 2564 feature_gate.go:328] unrecognized feature gate: Example2 May 21 11:30:45.428436 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426831 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 21 11:30:45.428436 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426833 2564 feature_gate.go:328] unrecognized feature gate: DualReplica May 21 11:30:45.428436 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426835 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 21 11:30:45.428436 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426838 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 21 11:30:45.428436 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426841 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 21 11:30:45.428436 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426843 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 21 11:30:45.428436 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426847 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 21 11:30:45.428436 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426849 2564 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 21 11:30:45.428436 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426852 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 21 11:30:45.428436 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.426856 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 21 11:30:45.428436 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427271 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 21 11:30:45.428436 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427277 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 21 11:30:45.428436 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427280 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 21 11:30:45.428436 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427283 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 21 11:30:45.428436 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427286 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 21 11:30:45.428436 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427288 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 21 11:30:45.428436 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427291 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 21 11:30:45.428436 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427294 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 21 11:30:45.428436 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427297 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 21 11:30:45.428904 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427299 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 21 11:30:45.428904 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427302 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 21 11:30:45.428904 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427304 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 21 11:30:45.428904 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427308 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 21 11:30:45.428904 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427310 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 21 11:30:45.428904 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427313 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 21 11:30:45.428904 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427315 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 21 11:30:45.428904 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427320 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 21 11:30:45.428904 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427323 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 21 11:30:45.428904 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427326 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 21 11:30:45.428904 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427329 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 21 11:30:45.428904 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427332 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 21 11:30:45.428904 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427335 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 21 11:30:45.428904 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427338 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 21 11:30:45.428904 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427341 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 21 11:30:45.428904 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427343 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 21 11:30:45.428904 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427345 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 21 11:30:45.428904 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427348 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 21 11:30:45.428904 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427351 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 21 11:30:45.429385 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427354 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 21 11:30:45.429385 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427356 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 21 11:30:45.429385 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427359 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 21 11:30:45.429385 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427362 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 21 11:30:45.429385 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427364 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 21 11:30:45.429385 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427367 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 21 11:30:45.429385 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427370 2564 feature_gate.go:328] unrecognized feature gate: Example2 May 21 11:30:45.429385 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427372 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 21 11:30:45.429385 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427374 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 21 11:30:45.429385 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427377 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 21 11:30:45.429385 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427379 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages May 21 11:30:45.429385 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427382 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 21 11:30:45.429385 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427384 2564 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 21 11:30:45.429385 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427387 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 21 11:30:45.429385 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427389 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 21 11:30:45.429385 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427392 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 21 11:30:45.429385 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427394 2564 feature_gate.go:328] unrecognized feature gate: NewOLM May 21 11:30:45.429385 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427397 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 21 11:30:45.429385 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427399 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 21 11:30:45.429385 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427401 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 21 11:30:45.429385 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427404 2564 feature_gate.go:328] unrecognized feature gate: Example May 21 11:30:45.429895 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427406 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 21 11:30:45.429895 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427409 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 21 11:30:45.429895 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427412 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 21 11:30:45.429895 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427414 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 21 11:30:45.429895 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427417 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 21 11:30:45.429895 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427419 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 21 11:30:45.429895 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427422 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 21 11:30:45.429895 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427424 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 21 11:30:45.429895 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427426 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 21 11:30:45.429895 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427429 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 21 11:30:45.429895 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427431 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 21 11:30:45.429895 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427434 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 21 11:30:45.429895 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427436 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 21 11:30:45.429895 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427438 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 21 11:30:45.429895 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427441 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 21 11:30:45.429895 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427443 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 21 11:30:45.429895 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427446 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 21 11:30:45.429895 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427448 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 21 11:30:45.429895 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427450 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 21 11:30:45.429895 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427453 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 21 11:30:45.430400 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427455 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 21 11:30:45.430400 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427458 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability May 21 11:30:45.430400 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427460 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 21 11:30:45.430400 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427463 2564 feature_gate.go:328] unrecognized feature gate: DualReplica May 21 11:30:45.430400 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427468 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 21 11:30:45.430400 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427470 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 21 11:30:45.430400 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427473 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 21 11:30:45.430400 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427475 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 21 11:30:45.430400 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427479 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 21 11:30:45.430400 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427483 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 21 11:30:45.430400 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427486 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 21 11:30:45.430400 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427489 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 21 11:30:45.430400 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427492 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores May 21 11:30:45.430400 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427494 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 21 11:30:45.430400 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427497 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 21 11:30:45.430400 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427499 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 21 11:30:45.430400 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.427502 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 21 11:30:45.430400 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.428814 2564 flags.go:64] FLAG: --address="0.0.0.0" May 21 11:30:45.430400 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.428825 2564 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" May 21 11:30:45.430400 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.428832 2564 flags.go:64] FLAG: --anonymous-auth="true" May 21 11:30:45.430400 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.428837 2564 flags.go:64] FLAG: --application-metrics-count-limit="100" May 21 11:30:45.430896 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.428841 2564 flags.go:64] FLAG: --authentication-token-webhook="false" May 21 11:30:45.430896 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.428845 2564 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" May 21 11:30:45.430896 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.428849 2564 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" May 21 11:30:45.430896 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.428854 2564 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" May 21 11:30:45.430896 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.428857 2564 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" May 21 11:30:45.430896 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.428860 2564 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" May 21 11:30:45.430896 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.428864 2564 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" May 21 11:30:45.430896 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.428867 2564 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" May 21 11:30:45.430896 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.428870 2564 flags.go:64] FLAG: --cgroup-driver="cgroupfs" May 21 11:30:45.430896 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.428873 2564 flags.go:64] FLAG: --cgroup-root="" May 21 11:30:45.430896 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.428876 2564 flags.go:64] FLAG: --cgroups-per-qos="true" May 21 11:30:45.430896 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.428879 2564 flags.go:64] FLAG: --client-ca-file="" May 21 11:30:45.430896 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.428881 2564 flags.go:64] FLAG: --cloud-config="" May 21 11:30:45.430896 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.428884 2564 flags.go:64] FLAG: --cloud-provider="external" May 21 11:30:45.430896 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.428887 2564 flags.go:64] FLAG: --cluster-dns="[]" May 21 11:30:45.430896 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.428891 2564 flags.go:64] FLAG: --cluster-domain="" May 21 11:30:45.430896 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.428894 2564 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" May 21 11:30:45.430896 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.428897 2564 flags.go:64] FLAG: --config-dir="" May 21 11:30:45.430896 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.428900 2564 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" May 21 11:30:45.430896 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.428904 2564 flags.go:64] FLAG: --container-log-max-files="5" May 21 11:30:45.430896 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.428908 2564 flags.go:64] FLAG: --container-log-max-size="10Mi" May 21 11:30:45.430896 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.428912 2564 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" May 21 11:30:45.430896 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.428915 2564 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" May 21 11:30:45.430896 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.428918 2564 flags.go:64] FLAG: --containerd-namespace="k8s.io" May 21 11:30:45.431591 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.428921 2564 flags.go:64] FLAG: --contention-profiling="false" May 21 11:30:45.431591 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.428924 2564 flags.go:64] FLAG: --cpu-cfs-quota="true" May 21 11:30:45.431591 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.428927 2564 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" May 21 11:30:45.431591 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.428930 2564 flags.go:64] FLAG: --cpu-manager-policy="none" May 21 11:30:45.431591 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.428933 2564 flags.go:64] FLAG: --cpu-manager-policy-options="" May 21 11:30:45.431591 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.428936 2564 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" May 21 11:30:45.431591 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.428939 2564 flags.go:64] FLAG: --enable-controller-attach-detach="true" May 21 11:30:45.431591 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.428942 2564 flags.go:64] FLAG: --enable-debugging-handlers="true" May 21 11:30:45.431591 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.428945 2564 flags.go:64] FLAG: --enable-load-reader="false" May 21 11:30:45.431591 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.428948 2564 flags.go:64] FLAG: --enable-server="true" May 21 11:30:45.431591 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.428951 2564 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" May 21 11:30:45.431591 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.428956 2564 flags.go:64] FLAG: --event-burst="100" May 21 11:30:45.431591 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.428959 2564 flags.go:64] FLAG: --event-qps="50" May 21 11:30:45.431591 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.428961 2564 flags.go:64] FLAG: --event-storage-age-limit="default=0" May 21 11:30:45.431591 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.428964 2564 flags.go:64] FLAG: --event-storage-event-limit="default=0" May 21 11:30:45.431591 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.428967 2564 flags.go:64] FLAG: --eviction-hard="" May 21 11:30:45.431591 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.428971 2564 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" May 21 11:30:45.431591 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.428973 2564 flags.go:64] FLAG: --eviction-minimum-reclaim="" May 21 11:30:45.431591 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.428976 2564 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" May 21 11:30:45.431591 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.428979 2564 flags.go:64] FLAG: --eviction-soft="" May 21 11:30:45.431591 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.428982 2564 flags.go:64] FLAG: --eviction-soft-grace-period="" May 21 11:30:45.431591 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.428984 2564 flags.go:64] FLAG: --exit-on-lock-contention="false" May 21 11:30:45.431591 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.428987 2564 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" May 21 11:30:45.431591 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.428990 2564 flags.go:64] FLAG: --experimental-mounter-path="" May 21 11:30:45.431591 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.428993 2564 flags.go:64] FLAG: --fail-cgroupv1="false" May 21 11:30:45.432203 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.428996 2564 flags.go:64] FLAG: --fail-swap-on="true" May 21 11:30:45.432203 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.428999 2564 flags.go:64] FLAG: --feature-gates="" May 21 11:30:45.432203 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429004 2564 flags.go:64] FLAG: --file-check-frequency="20s" May 21 11:30:45.432203 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429007 2564 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" May 21 11:30:45.432203 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429010 2564 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" May 21 11:30:45.432203 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429014 2564 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" May 21 11:30:45.432203 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429017 2564 flags.go:64] FLAG: --healthz-port="10248" May 21 11:30:45.432203 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429020 2564 flags.go:64] FLAG: --help="false" May 21 11:30:45.432203 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429023 2564 flags.go:64] FLAG: --hostname-override="ip-10-0-139-213.ec2.internal" May 21 11:30:45.432203 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429026 2564 flags.go:64] FLAG: --housekeeping-interval="10s" May 21 11:30:45.432203 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429029 2564 flags.go:64] FLAG: --http-check-frequency="20s" May 21 11:30:45.432203 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429032 2564 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" May 21 11:30:45.432203 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429035 2564 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" May 21 11:30:45.432203 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429038 2564 flags.go:64] FLAG: --image-gc-high-threshold="85" May 21 11:30:45.432203 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429041 2564 flags.go:64] FLAG: --image-gc-low-threshold="80" May 21 11:30:45.432203 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429044 2564 flags.go:64] FLAG: --image-service-endpoint="" May 21 11:30:45.432203 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429047 2564 flags.go:64] FLAG: --kernel-memcg-notification="false" May 21 11:30:45.432203 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429050 2564 flags.go:64] FLAG: --kube-api-burst="100" May 21 11:30:45.432203 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429052 2564 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" May 21 11:30:45.432203 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429056 2564 flags.go:64] FLAG: --kube-api-qps="50" May 21 11:30:45.432203 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429059 2564 flags.go:64] FLAG: --kube-reserved="" May 21 11:30:45.432203 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429061 2564 flags.go:64] FLAG: --kube-reserved-cgroup="" May 21 11:30:45.432203 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429064 2564 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" May 21 11:30:45.432203 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429068 2564 flags.go:64] FLAG: --kubelet-cgroups="" May 21 11:30:45.432809 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429070 2564 flags.go:64] FLAG: --local-storage-capacity-isolation="true" May 21 11:30:45.432809 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429085 2564 flags.go:64] FLAG: --lock-file="" May 21 11:30:45.432809 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429088 2564 flags.go:64] FLAG: --log-cadvisor-usage="false" May 21 11:30:45.432809 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429091 2564 flags.go:64] FLAG: --log-flush-frequency="5s" May 21 11:30:45.432809 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429094 2564 flags.go:64] FLAG: --log-json-info-buffer-size="0" May 21 11:30:45.432809 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429099 2564 flags.go:64] FLAG: --log-json-split-stream="false" May 21 11:30:45.432809 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429102 2564 flags.go:64] FLAG: --log-text-info-buffer-size="0" May 21 11:30:45.432809 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429106 2564 flags.go:64] FLAG: --log-text-split-stream="false" May 21 11:30:45.432809 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429109 2564 flags.go:64] FLAG: --logging-format="text" May 21 11:30:45.432809 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429111 2564 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" May 21 11:30:45.432809 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429115 2564 flags.go:64] FLAG: --make-iptables-util-chains="true" May 21 11:30:45.432809 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429118 2564 flags.go:64] FLAG: --manifest-url="" May 21 11:30:45.432809 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429121 2564 flags.go:64] FLAG: --manifest-url-header="" May 21 11:30:45.432809 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429126 2564 flags.go:64] FLAG: --max-housekeeping-interval="15s" May 21 11:30:45.432809 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429130 2564 flags.go:64] FLAG: --max-open-files="1000000" May 21 11:30:45.432809 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429134 2564 flags.go:64] FLAG: --max-pods="110" May 21 11:30:45.432809 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429137 2564 flags.go:64] FLAG: --maximum-dead-containers="-1" May 21 11:30:45.432809 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429140 2564 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" May 21 11:30:45.432809 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429143 2564 flags.go:64] FLAG: --memory-manager-policy="None" May 21 11:30:45.432809 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429146 2564 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" May 21 11:30:45.432809 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429149 2564 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" May 21 11:30:45.432809 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429152 2564 flags.go:64] FLAG: --node-ip="0.0.0.0" May 21 11:30:45.432809 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429155 2564 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" May 21 11:30:45.432809 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429162 2564 flags.go:64] FLAG: --node-status-max-images="50" May 21 11:30:45.432809 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429165 2564 flags.go:64] FLAG: --node-status-update-frequency="10s" May 21 11:30:45.433437 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429168 2564 flags.go:64] FLAG: --oom-score-adj="-999" May 21 11:30:45.433437 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429171 2564 flags.go:64] FLAG: --pod-cidr="" May 21 11:30:45.433437 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429173 2564 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:afe88671bf163c42b467d159dc6f0b784dbe28b1ed44e171edc9c06fd99cc6fc" May 21 11:30:45.433437 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429180 2564 flags.go:64] FLAG: --pod-manifest-path="" May 21 11:30:45.433437 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429182 2564 flags.go:64] FLAG: --pod-max-pids="-1" May 21 11:30:45.433437 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429186 2564 flags.go:64] FLAG: --pods-per-core="0" May 21 11:30:45.433437 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429188 2564 flags.go:64] FLAG: --port="10250" May 21 11:30:45.433437 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429191 2564 flags.go:64] FLAG: --protect-kernel-defaults="false" May 21 11:30:45.433437 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429194 2564 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0d3b526631f07ca3b" May 21 11:30:45.433437 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429197 2564 flags.go:64] FLAG: --qos-reserved="" May 21 11:30:45.433437 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429200 2564 flags.go:64] FLAG: --read-only-port="10255" May 21 11:30:45.433437 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429203 2564 flags.go:64] FLAG: --register-node="true" May 21 11:30:45.433437 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429206 2564 flags.go:64] FLAG: --register-schedulable="true" May 21 11:30:45.433437 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429209 2564 flags.go:64] FLAG: --register-with-taints="" May 21 11:30:45.433437 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429212 2564 flags.go:64] FLAG: --registry-burst="10" May 21 11:30:45.433437 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429215 2564 flags.go:64] FLAG: --registry-qps="5" May 21 11:30:45.433437 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429218 2564 flags.go:64] FLAG: --reserved-cpus="" May 21 11:30:45.433437 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429221 2564 flags.go:64] FLAG: --reserved-memory="" May 21 11:30:45.433437 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429225 2564 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" May 21 11:30:45.433437 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429228 2564 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" May 21 11:30:45.433437 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429231 2564 flags.go:64] FLAG: --rotate-certificates="false" May 21 11:30:45.433437 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429234 2564 flags.go:64] FLAG: --rotate-server-certificates="false" May 21 11:30:45.433437 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429238 2564 flags.go:64] FLAG: --runonce="false" May 21 11:30:45.433437 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429240 2564 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" May 21 11:30:45.433437 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429244 2564 flags.go:64] FLAG: --runtime-request-timeout="2m0s" May 21 11:30:45.434023 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429247 2564 flags.go:64] FLAG: --seccomp-default="false" May 21 11:30:45.434023 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429250 2564 flags.go:64] FLAG: --serialize-image-pulls="true" May 21 11:30:45.434023 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429252 2564 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" May 21 11:30:45.434023 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429255 2564 flags.go:64] FLAG: --storage-driver-db="cadvisor" May 21 11:30:45.434023 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429258 2564 flags.go:64] FLAG: --storage-driver-host="localhost:8086" May 21 11:30:45.434023 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429261 2564 flags.go:64] FLAG: --storage-driver-password="root" May 21 11:30:45.434023 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429265 2564 flags.go:64] FLAG: --storage-driver-secure="false" May 21 11:30:45.434023 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429267 2564 flags.go:64] FLAG: --storage-driver-table="stats" May 21 11:30:45.434023 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429270 2564 flags.go:64] FLAG: --storage-driver-user="root" May 21 11:30:45.434023 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429273 2564 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" May 21 11:30:45.434023 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429276 2564 flags.go:64] FLAG: --sync-frequency="1m0s" May 21 11:30:45.434023 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429278 2564 flags.go:64] FLAG: --system-cgroups="" May 21 11:30:45.434023 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429281 2564 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" May 21 11:30:45.434023 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429286 2564 flags.go:64] FLAG: --system-reserved-cgroup="" May 21 11:30:45.434023 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429289 2564 flags.go:64] FLAG: --tls-cert-file="" May 21 11:30:45.434023 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429292 2564 flags.go:64] FLAG: --tls-cipher-suites="[]" May 21 11:30:45.434023 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429296 2564 flags.go:64] FLAG: --tls-min-version="" May 21 11:30:45.434023 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429299 2564 flags.go:64] FLAG: --tls-private-key-file="" May 21 11:30:45.434023 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429301 2564 flags.go:64] FLAG: --topology-manager-policy="none" May 21 11:30:45.434023 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429306 2564 flags.go:64] FLAG: --topology-manager-policy-options="" May 21 11:30:45.434023 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429309 2564 flags.go:64] FLAG: --topology-manager-scope="container" May 21 11:30:45.434023 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429312 2564 flags.go:64] FLAG: --v="2" May 21 11:30:45.434023 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429316 2564 flags.go:64] FLAG: --version="false" May 21 11:30:45.434023 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429320 2564 flags.go:64] FLAG: --vmodule="" May 21 11:30:45.434023 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429324 2564 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" May 21 11:30:45.434667 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.429328 2564 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" May 21 11:30:45.434667 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429428 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 21 11:30:45.434667 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429432 2564 feature_gate.go:328] unrecognized feature gate: DualReplica May 21 11:30:45.434667 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429435 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 21 11:30:45.434667 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429438 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 21 11:30:45.434667 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429442 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 21 11:30:45.434667 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429445 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 21 11:30:45.434667 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429449 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 21 11:30:45.434667 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429451 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 21 11:30:45.434667 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429454 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 21 11:30:45.434667 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429456 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 21 11:30:45.434667 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429459 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 21 11:30:45.434667 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429462 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 21 11:30:45.434667 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429464 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 21 11:30:45.434667 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429466 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 21 11:30:45.434667 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429469 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 21 11:30:45.434667 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429471 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 21 11:30:45.434667 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429474 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 21 11:30:45.434667 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429476 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 21 11:30:45.434667 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429479 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 21 11:30:45.435174 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429481 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 21 11:30:45.435174 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429483 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 21 11:30:45.435174 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429486 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 21 11:30:45.435174 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429488 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 21 11:30:45.435174 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429491 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 21 11:30:45.435174 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429494 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 21 11:30:45.435174 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429498 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 21 11:30:45.435174 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429500 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 21 11:30:45.435174 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429503 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 21 11:30:45.435174 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429505 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 21 11:30:45.435174 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429508 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 21 11:30:45.435174 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429510 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 21 11:30:45.435174 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429514 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 21 11:30:45.435174 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429517 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 21 11:30:45.435174 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429520 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 21 11:30:45.435174 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429523 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 21 11:30:45.435174 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429525 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 21 11:30:45.435174 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429528 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 21 11:30:45.435174 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429531 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 21 11:30:45.435669 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429533 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 21 11:30:45.435669 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429536 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 21 11:30:45.435669 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429539 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 21 11:30:45.435669 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429541 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 21 11:30:45.435669 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429543 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 21 11:30:45.435669 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429546 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 21 11:30:45.435669 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429548 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 21 11:30:45.435669 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429550 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 21 11:30:45.435669 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429553 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 21 11:30:45.435669 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429555 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 21 11:30:45.435669 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429558 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 21 11:30:45.435669 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429560 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 21 11:30:45.435669 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429562 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 21 11:30:45.435669 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429565 2564 feature_gate.go:328] unrecognized feature gate: Example May 21 11:30:45.435669 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429569 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 21 11:30:45.435669 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429572 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 21 11:30:45.435669 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429575 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 21 11:30:45.435669 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429578 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 21 11:30:45.435669 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429580 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 21 11:30:45.436152 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429584 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 21 11:30:45.436152 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429587 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 21 11:30:45.436152 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429590 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 21 11:30:45.436152 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429592 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 21 11:30:45.436152 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429595 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability May 21 11:30:45.436152 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429597 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 21 11:30:45.436152 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429602 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 21 11:30:45.436152 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429604 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 21 11:30:45.436152 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429607 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 21 11:30:45.436152 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429609 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 21 11:30:45.436152 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429612 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 21 11:30:45.436152 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429614 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 21 11:30:45.436152 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429617 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages May 21 11:30:45.436152 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429620 2564 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 21 11:30:45.436152 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429622 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 21 11:30:45.436152 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429625 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores May 21 11:30:45.436152 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429628 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 21 11:30:45.436152 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429630 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 21 11:30:45.436152 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429632 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 21 11:30:45.436152 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429635 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 21 11:30:45.436649 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429637 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 21 11:30:45.436649 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429639 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 21 11:30:45.436649 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429642 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 21 11:30:45.436649 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429644 2564 feature_gate.go:328] unrecognized feature gate: Example2 May 21 11:30:45.436649 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429647 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 21 11:30:45.436649 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429649 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 21 11:30:45.436649 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429651 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 21 11:30:45.436649 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429654 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 21 11:30:45.436649 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.429657 2564 feature_gate.go:328] unrecognized feature gate: NewOLM May 21 11:30:45.436649 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.430268 2564 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} May 21 11:30:45.439713 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.439689 2564 server.go:530] "Kubelet version" kubeletVersion="v1.33.11" May 21 11:30:45.439713 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.439711 2564 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" May 21 11:30:45.439884 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.439780 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 21 11:30:45.439884 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.439788 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 21 11:30:45.439884 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.439792 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 21 11:30:45.439884 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.439796 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 21 11:30:45.439884 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.439801 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 21 11:30:45.439884 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.439805 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 21 11:30:45.439884 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.439811 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 21 11:30:45.439884 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.439815 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 21 11:30:45.439884 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.439819 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 21 11:30:45.439884 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.439824 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 21 11:30:45.439884 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.439830 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 21 11:30:45.439884 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.439835 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 21 11:30:45.439884 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.439839 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 21 11:30:45.439884 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.439843 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 21 11:30:45.439884 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.439847 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 21 11:30:45.439884 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.439852 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 21 11:30:45.439884 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.439856 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 21 11:30:45.439884 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.439860 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 21 11:30:45.439884 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.439865 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 21 11:30:45.440724 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.439869 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 21 11:30:45.440724 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.439873 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability May 21 11:30:45.440724 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.439877 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 21 11:30:45.440724 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.439880 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 21 11:30:45.440724 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.439884 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 21 11:30:45.440724 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.439888 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 21 11:30:45.440724 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.439893 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 21 11:30:45.440724 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.439897 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 21 11:30:45.440724 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.439902 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 21 11:30:45.440724 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.439905 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 21 11:30:45.440724 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.439910 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 21 11:30:45.440724 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.439914 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 21 11:30:45.440724 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.439918 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 21 11:30:45.440724 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.439922 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 21 11:30:45.440724 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.439926 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 21 11:30:45.440724 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.439930 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 21 11:30:45.440724 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.439934 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 21 11:30:45.440724 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.439938 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 21 11:30:45.440724 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.439943 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 21 11:30:45.441325 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.439949 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 21 11:30:45.441325 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.439956 2564 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 21 11:30:45.441325 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.439960 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 21 11:30:45.441325 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.439965 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 21 11:30:45.441325 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.439969 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 21 11:30:45.441325 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.439973 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 21 11:30:45.441325 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.439977 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 21 11:30:45.441325 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.439981 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 21 11:30:45.441325 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.439985 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 21 11:30:45.441325 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.439990 2564 feature_gate.go:328] unrecognized feature gate: NewOLM May 21 11:30:45.441325 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.439993 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 21 11:30:45.441325 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.439997 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 21 11:30:45.441325 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440001 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 21 11:30:45.441325 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440006 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 21 11:30:45.441325 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440011 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 21 11:30:45.441325 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440016 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 21 11:30:45.441325 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440020 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 21 11:30:45.441325 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440025 2564 feature_gate.go:328] unrecognized feature gate: DualReplica May 21 11:30:45.441325 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440029 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 21 11:30:45.441325 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440033 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores May 21 11:30:45.441891 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440038 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 21 11:30:45.441891 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440042 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 21 11:30:45.441891 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440046 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 21 11:30:45.441891 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440051 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 21 11:30:45.441891 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440055 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 21 11:30:45.441891 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440059 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 21 11:30:45.441891 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440063 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages May 21 11:30:45.441891 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440067 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 21 11:30:45.441891 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440071 2564 feature_gate.go:328] unrecognized feature gate: Example May 21 11:30:45.441891 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440097 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 21 11:30:45.441891 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440101 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 21 11:30:45.441891 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440106 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 21 11:30:45.441891 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440110 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 21 11:30:45.441891 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440114 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 21 11:30:45.441891 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440118 2564 feature_gate.go:328] unrecognized feature gate: Example2 May 21 11:30:45.441891 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440122 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 21 11:30:45.441891 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440126 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 21 11:30:45.441891 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440130 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 21 11:30:45.441891 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440135 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 21 11:30:45.441891 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440140 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 21 11:30:45.442646 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440144 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 21 11:30:45.442646 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440148 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 21 11:30:45.442646 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440152 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 21 11:30:45.442646 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440156 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 21 11:30:45.442646 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440160 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 21 11:30:45.442646 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440165 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 21 11:30:45.442646 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440170 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 21 11:30:45.442646 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440174 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 21 11:30:45.442646 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.440182 2564 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} May 21 11:30:45.442646 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440338 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 21 11:30:45.442646 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440346 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 21 11:30:45.442646 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440351 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 21 11:30:45.442646 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440355 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 21 11:30:45.442646 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440359 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 21 11:30:45.442646 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440364 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 21 11:30:45.443034 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440368 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 21 11:30:45.443034 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440373 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores May 21 11:30:45.443034 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440377 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 21 11:30:45.443034 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440381 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability May 21 11:30:45.443034 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440385 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 21 11:30:45.443034 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440389 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 21 11:30:45.443034 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440393 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 21 11:30:45.443034 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440397 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 21 11:30:45.443034 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440401 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 21 11:30:45.443034 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440405 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 21 11:30:45.443034 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440410 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 21 11:30:45.443034 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440414 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 21 11:30:45.443034 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440418 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 21 11:30:45.443034 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440422 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 21 11:30:45.443034 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440426 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 21 11:30:45.443034 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440430 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 21 11:30:45.443034 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440436 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 21 11:30:45.443034 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440441 2564 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 21 11:30:45.443034 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440445 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 21 11:30:45.443534 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440449 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 21 11:30:45.443534 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440453 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 21 11:30:45.443534 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440457 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 21 11:30:45.443534 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440462 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 21 11:30:45.443534 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440467 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 21 11:30:45.443534 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440471 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 21 11:30:45.443534 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440475 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 21 11:30:45.443534 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440479 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 21 11:30:45.443534 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440483 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 21 11:30:45.443534 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440487 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 21 11:30:45.443534 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440491 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 21 11:30:45.443534 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440495 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 21 11:30:45.443534 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440502 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 21 11:30:45.443534 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440507 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 21 11:30:45.443534 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440512 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 21 11:30:45.443534 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440517 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 21 11:30:45.443534 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440522 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 21 11:30:45.443534 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440526 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 21 11:30:45.443534 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440531 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 21 11:30:45.443988 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440536 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 21 11:30:45.443988 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440540 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 21 11:30:45.443988 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440545 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 21 11:30:45.443988 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440549 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 21 11:30:45.443988 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440553 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 21 11:30:45.443988 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440557 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 21 11:30:45.443988 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440561 2564 feature_gate.go:328] unrecognized feature gate: DualReplica May 21 11:30:45.443988 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440565 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 21 11:30:45.443988 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440569 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 21 11:30:45.443988 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440573 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 21 11:30:45.443988 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440577 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 21 11:30:45.443988 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440582 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 21 11:30:45.443988 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440588 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 21 11:30:45.443988 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440593 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 21 11:30:45.443988 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440598 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 21 11:30:45.443988 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440601 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 21 11:30:45.443988 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440605 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 21 11:30:45.443988 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440610 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 21 11:30:45.443988 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440614 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 21 11:30:45.444479 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440618 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 21 11:30:45.444479 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440622 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 21 11:30:45.444479 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440626 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 21 11:30:45.444479 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440630 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 21 11:30:45.444479 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440634 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages May 21 11:30:45.444479 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440638 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 21 11:30:45.444479 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440642 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 21 11:30:45.444479 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440646 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 21 11:30:45.444479 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440650 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 21 11:30:45.444479 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440654 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 21 11:30:45.444479 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440658 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 21 11:30:45.444479 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440662 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 21 11:30:45.444479 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440666 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 21 11:30:45.444479 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440670 2564 feature_gate.go:328] unrecognized feature gate: Example May 21 11:30:45.444479 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440674 2564 feature_gate.go:328] unrecognized feature gate: Example2 May 21 11:30:45.444479 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440679 2564 feature_gate.go:328] unrecognized feature gate: NewOLM May 21 11:30:45.444479 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440682 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 21 11:30:45.444479 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440686 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 21 11:30:45.444479 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440690 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 21 11:30:45.444479 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440695 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 21 11:30:45.444987 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440698 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 21 11:30:45.444987 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440703 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 21 11:30:45.444987 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:45.440707 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 21 11:30:45.444987 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.440715 2564 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} May 21 11:30:45.444987 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.441455 2564 server.go:962] "Client rotation is on, will bootstrap in background" May 21 11:30:45.446264 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.446246 2564 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" May 21 11:30:45.447357 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.447343 2564 server.go:1019] "Starting client certificate rotation" May 21 11:30:45.447462 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.447444 2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" May 21 11:30:45.447498 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.447485 2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" May 21 11:30:45.472652 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.472629 2564 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" May 21 11:30:45.474529 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.474509 2564 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" May 21 11:30:45.493007 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.492991 2564 log.go:25] "Validated CRI v1 runtime API" May 21 11:30:45.499687 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.499664 2564 log.go:25] "Validated CRI v1 image API" May 21 11:30:45.500805 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.500787 2564 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" May 21 11:30:45.504641 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.504625 2564 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" May 21 11:30:45.506089 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.506058 2564 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 864ae8e9-5ba9-42e7-a812-194c2b219680:/dev/nvme0n1p4 b47d8f62-d034-4c65-8d73-37b3f9cc9123:/dev/nvme0n1p3] May 21 11:30:45.506148 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.506093 2564 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] May 21 11:30:45.510359 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.510262 2564 manager.go:217] Machine: {Timestamp:2026-05-21 11:30:45.509725967 +0000 UTC m=+0.405009503 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099967 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f4919c4870b6df73b5b8a62798982 SystemUUID:ec2f4919-c487-0b6d-f73b-5b8a62798982 BootID:686fa09a-004a-42cd-bf0a-b495e65f6715 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:43:f6:3f:b6:7f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:43:f6:3f:b6:7f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:d6:83:4e:80:97:24 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} May 21 11:30:45.512643 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.512632 2564 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. May 21 11:30:45.512732 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.512720 2564 manager.go:233] Version: {KernelVersion:5.14.0-570.114.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260512-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} May 21 11:30:45.515195 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.515173 2564 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] May 21 11:30:45.515339 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.515198 2564 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-213.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} May 21 11:30:45.515384 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.515349 2564 topology_manager.go:138] "Creating topology manager with none policy" May 21 11:30:45.515384 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.515358 2564 container_manager_linux.go:306] "Creating device plugin manager" May 21 11:30:45.515384 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.515366 2564 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" May 21 11:30:45.515384 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.515378 2564 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" May 21 11:30:45.516793 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.516782 2564 state_mem.go:36] "Initialized new in-memory state store" May 21 11:30:45.517059 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.517049 2564 server.go:1267] "Using root directory" path="/var/lib/kubelet" May 21 11:30:45.519411 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.519399 2564 kubelet.go:491] "Attempting to sync node with API server" May 21 11:30:45.519447 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.519423 2564 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests" May 21 11:30:45.519447 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.519437 2564 file.go:69] "Watching path" path="/etc/kubernetes/manifests" May 21 11:30:45.519447 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.519446 2564 kubelet.go:397] "Adding apiserver pod source" May 21 11:30:45.519534 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.519456 2564 apiserver.go:42] "Waiting for node sync before watching apiserver pods" May 21 11:30:45.520639 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.520624 2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" May 21 11:30:45.520678 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.520655 2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" May 21 11:30:45.523665 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.523647 2564 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.11-2.rhaos4.20.gitb2a8320.el9" apiVersion="v1" May 21 11:30:45.526445 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.526431 2564 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" May 21 11:30:45.528591 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.528577 2564 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" May 21 11:30:45.528674 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.528595 2564 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" May 21 11:30:45.528674 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.528602 2564 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" May 21 11:30:45.528674 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.528611 2564 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path" May 21 11:30:45.528674 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.528624 2564 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs" May 21 11:30:45.528674 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.528634 2564 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret" May 21 11:30:45.528674 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.528640 2564 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" May 21 11:30:45.528674 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.528645 2564 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" May 21 11:30:45.528674 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.528653 2564 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc" May 21 11:30:45.528674 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.528659 2564 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap" May 21 11:30:45.528674 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.528669 2564 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected" May 21 11:30:45.528674 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.528674 2564 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" May 21 11:30:45.530190 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.530167 2564 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi" May 21 11:30:45.530190 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.530193 2564 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image" May 21 11:30:45.530470 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:45.530450 2564 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-213.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" May 21 11:30:45.530591 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:45.530520 2564 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" May 21 11:30:45.532145 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.532127 2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vcx7r" May 21 11:30:45.534228 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.534215 2564 watchdog_linux.go:99] "Systemd watchdog is not enabled" May 21 11:30:45.534293 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.534251 2564 server.go:1295] "Started kubelet" May 21 11:30:45.534348 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.534289 2564 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 May 21 11:30:45.534502 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.534435 2564 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 May 21 11:30:45.534555 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.534545 2564 server_v1.go:47] "podresources" method="list" useActivePods=true May 21 11:30:45.535241 ip-10-0-139-213 systemd[1]: Started Kubernetes Kubelet. May 21 11:30:45.535901 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.535705 2564 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" May 21 11:30:45.538042 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.538024 2564 server.go:317] "Adding debug handlers to kubelet server" May 21 11:30:45.538582 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.538560 2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vcx7r" May 21 11:30:45.542792 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.542774 2564 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-213.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope May 21 11:30:45.543485 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:45.542546 2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-213.ec2.internal.18b191432f865a57 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-213.ec2.internal,UID:ip-10-0-139-213.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-213.ec2.internal,},FirstTimestamp:2026-05-21 11:30:45.534227031 +0000 UTC m=+0.429510567,LastTimestamp:2026-05-21 11:30:45.534227031 +0000 UTC m=+0.429510567,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-213.ec2.internal,}" May 21 11:30:45.543581 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.543534 2564 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" May 21 11:30:45.543641 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.543611 2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" May 21 11:30:45.543770 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.543749 2564 volume_manager.go:295] "The desired_state_of_world populator starts" May 21 11:30:45.543770 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.543769 2564 volume_manager.go:297] "Starting Kubelet Volume Manager" May 21 11:30:45.543896 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.543876 2564 desired_state_of_world_populator.go:150] "Desired state populator starts to run" May 21 11:30:45.543974 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.543918 2564 reconstruct.go:97] "Volume reconstruction finished" May 21 11:30:45.543974 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.543926 2564 reconciler.go:26] "Reconciler: start to sync state" May 21 11:30:45.544573 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:45.544551 2564 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-213.ec2.internal\" not found" May 21 11:30:45.544645 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:45.544592 2564 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" May 21 11:30:45.544818 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.544804 2564 factory.go:55] Registering systemd factory May 21 11:30:45.544898 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.544831 2564 factory.go:223] Registration of the systemd container factory successfully May 21 11:30:45.545051 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.545038 2564 factory.go:153] Registering CRI-O factory May 21 11:30:45.545111 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.545055 2564 factory.go:223] Registration of the crio container factory successfully May 21 11:30:45.545149 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.545138 2564 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory May 21 11:30:45.545179 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.545160 2564 factory.go:103] Registering Raw factory May 21 11:30:45.545179 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.545176 2564 manager.go:1196] Started watching for new ooms in manager May 21 11:30:45.545610 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.545598 2564 manager.go:319] Starting recovery of all containers May 21 11:30:45.552004 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.551853 2564 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" May 21 11:30:45.554053 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.554013 2564 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6" May 21 11:30:45.555037 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:45.555012 2564 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-139-213.ec2.internal\" not found" node="ip-10-0-139-213.ec2.internal" May 21 11:30:45.557395 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.557377 2564 manager.go:324] Recovery completed May 21 11:30:45.562297 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.562285 2564 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 21 11:30:45.564762 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.564747 2564 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-213.ec2.internal" event="NodeHasSufficientMemory" May 21 11:30:45.564825 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.564777 2564 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-213.ec2.internal" event="NodeHasNoDiskPressure" May 21 11:30:45.564825 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.564788 2564 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-213.ec2.internal" event="NodeHasSufficientPID" May 21 11:30:45.565951 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.565933 2564 cpu_manager.go:222] "Starting CPU manager" policy="none" May 21 11:30:45.565951 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.565950 2564 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" May 21 11:30:45.566096 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.565986 2564 state_mem.go:36] "Initialized new in-memory state store" May 21 11:30:45.568303 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.568289 2564 policy_none.go:49] "None policy: Start" May 21 11:30:45.568368 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.568307 2564 memory_manager.go:186] "Starting memorymanager" policy="None" May 21 11:30:45.568368 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.568318 2564 state_mem.go:35] "Initializing new in-memory state store" May 21 11:30:45.603674 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.603652 2564 manager.go:341] "Starting Device Plugin manager" May 21 11:30:45.619250 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:45.603701 2564 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" May 21 11:30:45.619250 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.603716 2564 server.go:85] "Starting device plugin registration server" May 21 11:30:45.619250 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.603994 2564 eviction_manager.go:189] "Eviction manager: starting control loop" May 21 11:30:45.619250 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.604005 2564 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" May 21 11:30:45.619250 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.604121 2564 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" May 21 11:30:45.619250 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.604216 2564 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" May 21 11:30:45.619250 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.604228 2564 plugin_manager.go:118] "Starting Kubelet Plugin Manager" May 21 11:30:45.619250 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:45.604838 2564 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" May 21 11:30:45.619250 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:45.604868 2564 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-213.ec2.internal\" not found" May 21 11:30:45.678935 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.678863 2564 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4" May 21 11:30:45.678935 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.678892 2564 status_manager.go:230] "Starting to sync pod status with apiserver" May 21 11:30:45.678935 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.678910 2564 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." May 21 11:30:45.678935 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.678917 2564 kubelet.go:2451] "Starting kubelet main sync loop" May 21 11:30:45.679224 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:45.678952 2564 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" May 21 11:30:45.682855 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.682836 2564 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" May 21 11:30:45.705058 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.705033 2564 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 21 11:30:45.706203 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.706175 2564 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-213.ec2.internal" event="NodeHasSufficientMemory" May 21 11:30:45.706300 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.706215 2564 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-213.ec2.internal" event="NodeHasNoDiskPressure" May 21 11:30:45.706300 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.706232 2564 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-213.ec2.internal" event="NodeHasSufficientPID" May 21 11:30:45.706300 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.706260 2564 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-213.ec2.internal" May 21 11:30:45.715857 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.715836 2564 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-213.ec2.internal" May 21 11:30:45.715857 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:45.715859 2564 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-213.ec2.internal\": node \"ip-10-0-139-213.ec2.internal\" not found" May 21 11:30:45.730129 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:45.730109 2564 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-213.ec2.internal\" not found" May 21 11:30:45.779443 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.779408 2564 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-139-213.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-213.ec2.internal"] May 21 11:30:45.779524 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.779488 2564 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 21 11:30:45.780812 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.780795 2564 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-213.ec2.internal" event="NodeHasSufficientMemory" May 21 11:30:45.780891 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.780827 2564 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-213.ec2.internal" event="NodeHasNoDiskPressure" May 21 11:30:45.780891 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.780839 2564 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-213.ec2.internal" event="NodeHasSufficientPID" May 21 11:30:45.783141 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.783127 2564 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 21 11:30:45.783277 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.783263 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-213.ec2.internal" May 21 11:30:45.783362 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.783289 2564 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 21 11:30:45.783823 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.783803 2564 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-213.ec2.internal" event="NodeHasSufficientMemory" May 21 11:30:45.783923 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.783837 2564 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-213.ec2.internal" event="NodeHasNoDiskPressure" May 21 11:30:45.783923 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.783852 2564 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-213.ec2.internal" event="NodeHasSufficientPID" May 21 11:30:45.783923 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.783917 2564 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-213.ec2.internal" event="NodeHasSufficientMemory" May 21 11:30:45.784018 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.783946 2564 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-213.ec2.internal" event="NodeHasNoDiskPressure" May 21 11:30:45.784018 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.783958 2564 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-213.ec2.internal" event="NodeHasSufficientPID" May 21 11:30:45.786200 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.786176 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-213.ec2.internal" May 21 11:30:45.786275 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.786210 2564 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 21 11:30:45.786837 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.786821 2564 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-213.ec2.internal" event="NodeHasSufficientMemory" May 21 11:30:45.786896 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.786853 2564 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-213.ec2.internal" event="NodeHasNoDiskPressure" May 21 11:30:45.786896 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.786867 2564 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-213.ec2.internal" event="NodeHasSufficientPID" May 21 11:30:45.808808 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:45.808786 2564 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-213.ec2.internal\" not found" node="ip-10-0-139-213.ec2.internal" May 21 11:30:45.813392 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:45.813376 2564 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-213.ec2.internal\" not found" node="ip-10-0-139-213.ec2.internal" May 21 11:30:45.830446 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:45.830430 2564 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-213.ec2.internal\" not found" May 21 11:30:45.846266 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.846249 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3da4ba4cdfe6fc200fbc8989e0319648-config\") pod \"kube-apiserver-proxy-ip-10-0-139-213.ec2.internal\" (UID: \"3da4ba4cdfe6fc200fbc8989e0319648\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-213.ec2.internal" May 21 11:30:45.846359 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.846275 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0694ef6a59bf20f80056d0d4f3f2967d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-213.ec2.internal\" (UID: \"0694ef6a59bf20f80056d0d4f3f2967d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-213.ec2.internal" May 21 11:30:45.846359 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.846295 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0694ef6a59bf20f80056d0d4f3f2967d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-213.ec2.internal\" (UID: \"0694ef6a59bf20f80056d0d4f3f2967d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-213.ec2.internal" May 21 11:30:45.930612 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:45.930536 2564 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-213.ec2.internal\" not found" May 21 11:30:45.947440 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.947414 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3da4ba4cdfe6fc200fbc8989e0319648-config\") pod \"kube-apiserver-proxy-ip-10-0-139-213.ec2.internal\" (UID: \"3da4ba4cdfe6fc200fbc8989e0319648\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-213.ec2.internal" May 21 11:30:45.947440 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.947429 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3da4ba4cdfe6fc200fbc8989e0319648-config\") pod \"kube-apiserver-proxy-ip-10-0-139-213.ec2.internal\" (UID: \"3da4ba4cdfe6fc200fbc8989e0319648\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-213.ec2.internal" May 21 11:30:45.947573 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.947454 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0694ef6a59bf20f80056d0d4f3f2967d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-213.ec2.internal\" (UID: \"0694ef6a59bf20f80056d0d4f3f2967d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-213.ec2.internal" May 21 11:30:45.947573 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.947475 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0694ef6a59bf20f80056d0d4f3f2967d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-213.ec2.internal\" (UID: \"0694ef6a59bf20f80056d0d4f3f2967d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-213.ec2.internal" May 21 11:30:45.947573 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.947510 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0694ef6a59bf20f80056d0d4f3f2967d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-213.ec2.internal\" (UID: \"0694ef6a59bf20f80056d0d4f3f2967d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-213.ec2.internal" May 21 11:30:45.947573 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:45.947550 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0694ef6a59bf20f80056d0d4f3f2967d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-213.ec2.internal\" (UID: \"0694ef6a59bf20f80056d0d4f3f2967d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-213.ec2.internal" May 21 11:30:46.030889 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:46.030840 2564 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-213.ec2.internal\" not found" May 21 11:30:46.111372 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.111340 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-213.ec2.internal" May 21 11:30:46.115844 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.115824 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-213.ec2.internal" May 21 11:30:46.131571 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:46.131537 2564 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-213.ec2.internal\" not found" May 21 11:30:46.232144 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:46.232039 2564 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-213.ec2.internal\" not found" May 21 11:30:46.332552 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:46.332518 2564 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-213.ec2.internal\" not found" May 21 11:30:46.422094 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.422054 2564 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" May 21 11:30:46.443761 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.443728 2564 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-213.ec2.internal" May 21 11:30:46.446958 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.446939 2564 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" May 21 11:30:46.447109 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.447069 2564 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" May 21 11:30:46.447109 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.447102 2564 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" May 21 11:30:46.447238 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.447140 2564 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" May 21 11:30:46.447238 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:46.447141 2564 kubelet.go:3342] "Failed creating a mirror pod" err="Post \"https://a148dd3d041544c53ae054e65d0f64d8-6ab7ca56d48c0657.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/kube-system/pods\": read tcp 10.0.139.213:39424->52.44.224.246:6443: use of closed network connection" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-213.ec2.internal" May 21 11:30:46.447238 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.447166 2564 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-213.ec2.internal" May 21 11:30:46.468511 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.468493 2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" May 21 11:30:46.520206 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.520124 2564 apiserver.go:52] "Watching apiserver" May 21 11:30:46.528445 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.528420 2564 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" May 21 11:30:46.530415 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.530391 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2l94","openshift-cluster-node-tuning-operator/tuned-lbcnh","openshift-image-registry/node-ca-29k74","openshift-multus/multus-additional-cni-plugins-9nh44","openshift-multus/multus-q4k9w","openshift-network-operator/iptables-alerter-btjpv","openshift-ovn-kubernetes/ovnkube-node-sr2wt","openshift-dns/node-resolver-lkfnm","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-213.ec2.internal","openshift-multus/network-metrics-daemon-xf5gc","openshift-network-diagnostics/network-check-target-vw6wd","kube-system/konnectivity-agent-w8vhb"] May 21 11:30:46.534959 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.534931 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2l94" May 21 11:30:46.535804 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.535758 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-lbcnh" May 21 11:30:46.537636 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.537616 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-29k74" May 21 11:30:46.538385 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.538367 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" May 21 11:30:46.538685 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.538670 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" May 21 11:30:46.538685 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.538677 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" May 21 11:30:46.538819 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.538688 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" May 21 11:30:46.538819 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.538759 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" May 21 11:30:46.538910 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.538857 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-wm588\"" May 21 11:30:46.538910 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.538893 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-5hgjj\"" May 21 11:30:46.540341 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.540325 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" May 21 11:30:46.540438 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.540345 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-ng8rg\"" May 21 11:30:46.540556 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.540538 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" May 21 11:30:46.540623 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.540575 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9nh44" May 21 11:30:46.541102 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.541087 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" May 21 11:30:46.542141 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.542117 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-05-20 11:25:45 +0000 UTC" deadline="2027-10-18 08:33:56.323990606 +0000 UTC" May 21 11:30:46.542212 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.542141 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12357h3m9.781852589s" May 21 11:30:46.543015 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.542999 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-q4k9w" May 21 11:30:46.543130 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.543114 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" May 21 11:30:46.543225 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.543155 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-6d5kp\"" May 21 11:30:46.543289 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.543239 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" May 21 11:30:46.543433 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.543418 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" May 21 11:30:46.543508 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.543446 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" May 21 11:30:46.543563 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.543547 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" May 21 11:30:46.544653 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.544639 2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" May 21 11:30:46.545418 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.545405 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-btjpv" May 21 11:30:46.545629 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.545597 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-9fjds\"" May 21 11:30:46.545728 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.545648 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" May 21 11:30:46.547652 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.547634 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.548071 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.548053 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" May 21 11:30:46.548071 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.548057 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" May 21 11:30:46.548211 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.548062 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-khwxx\"" May 21 11:30:46.548211 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.548161 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" May 21 11:30:46.550028 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.550010 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" May 21 11:30:46.550134 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.550065 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0fd57831-cff3-4f96-8695-cb4d02bc95a9-serviceca\") pod \"node-ca-29k74\" (UID: \"0fd57831-cff3-4f96-8695-cb4d02bc95a9\") " pod="openshift-image-registry/node-ca-29k74" May 21 11:30:46.550134 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.550114 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-jg429\"" May 21 11:30:46.550134 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.550118 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bfa04605-9b0c-4c9b-a808-598623c60d2d-cni-binary-copy\") pod \"multus-additional-cni-plugins-9nh44\" (UID: \"bfa04605-9b0c-4c9b-a808-598623c60d2d\") " pod="openshift-multus/multus-additional-cni-plugins-9nh44" May 21 11:30:46.550362 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.550145 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bfa04605-9b0c-4c9b-a808-598623c60d2d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9nh44\" (UID: \"bfa04605-9b0c-4c9b-a808-598623c60d2d\") " pod="openshift-multus/multus-additional-cni-plugins-9nh44" May 21 11:30:46.550362 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.550171 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lkfnm" May 21 11:30:46.550362 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.550207 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bplk\" (UniqueName: \"kubernetes.io/projected/bfa04605-9b0c-4c9b-a808-598623c60d2d-kube-api-access-4bplk\") pod \"multus-additional-cni-plugins-9nh44\" (UID: \"bfa04605-9b0c-4c9b-a808-598623c60d2d\") " pod="openshift-multus/multus-additional-cni-plugins-9nh44" May 21 11:30:46.550362 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.550227 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" May 21 11:30:46.550362 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.550242 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fceead7c-3987-4250-816f-e89182d3b315-multus-cni-dir\") pod \"multus-q4k9w\" (UID: \"fceead7c-3987-4250-816f-e89182d3b315\") " pod="openshift-multus/multus-q4k9w" May 21 11:30:46.550362 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.550274 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fceead7c-3987-4250-816f-e89182d3b315-multus-socket-dir-parent\") pod \"multus-q4k9w\" (UID: \"fceead7c-3987-4250-816f-e89182d3b315\") " pod="openshift-multus/multus-q4k9w" May 21 11:30:46.550362 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.550322 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfxxv\" (UniqueName: \"kubernetes.io/projected/b3fb34bd-0e60-47f6-9fa6-2bd541ce9deb-kube-api-access-hfxxv\") pod \"iptables-alerter-btjpv\" (UID: \"b3fb34bd-0e60-47f6-9fa6-2bd541ce9deb\") " pod="openshift-network-operator/iptables-alerter-btjpv" May 21 11:30:46.550362 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.550358 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7qs9\" (UniqueName: \"kubernetes.io/projected/5516274d-6162-4809-bfc0-1f5d4b850794-kube-api-access-x7qs9\") pod \"aws-ebs-csi-driver-node-m2l94\" (UID: \"5516274d-6162-4809-bfc0-1f5d4b850794\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2l94" May 21 11:30:46.550746 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.550389 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e0c35189-f247-439b-9a5e-d199d3963920-host\") pod \"tuned-lbcnh\" (UID: \"e0c35189-f247-439b-9a5e-d199d3963920\") " pod="openshift-cluster-node-tuning-operator/tuned-lbcnh" May 21 11:30:46.550746 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.550421 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmkdw\" (UniqueName: \"kubernetes.io/projected/0fd57831-cff3-4f96-8695-cb4d02bc95a9-kube-api-access-kmkdw\") pod \"node-ca-29k74\" (UID: \"0fd57831-cff3-4f96-8695-cb4d02bc95a9\") " pod="openshift-image-registry/node-ca-29k74" May 21 11:30:46.550746 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.550443 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" May 21 11:30:46.550746 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.550448 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fceead7c-3987-4250-816f-e89182d3b315-host-run-netns\") pod \"multus-q4k9w\" (UID: \"fceead7c-3987-4250-816f-e89182d3b315\") " pod="openshift-multus/multus-q4k9w" May 21 11:30:46.550746 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.550480 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fceead7c-3987-4250-816f-e89182d3b315-host-var-lib-cni-bin\") pod \"multus-q4k9w\" (UID: \"fceead7c-3987-4250-816f-e89182d3b315\") " pod="openshift-multus/multus-q4k9w" May 21 11:30:46.550746 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.550507 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fceead7c-3987-4250-816f-e89182d3b315-multus-conf-dir\") pod \"multus-q4k9w\" (UID: \"fceead7c-3987-4250-816f-e89182d3b315\") " pod="openshift-multus/multus-q4k9w" May 21 11:30:46.550746 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.550652 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fceead7c-3987-4250-816f-e89182d3b315-etc-kubernetes\") pod \"multus-q4k9w\" (UID: \"fceead7c-3987-4250-816f-e89182d3b315\") " pod="openshift-multus/multus-q4k9w" May 21 11:30:46.551016 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.550696 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b3fb34bd-0e60-47f6-9fa6-2bd541ce9deb-iptables-alerter-script\") pod \"iptables-alerter-btjpv\" (UID: \"b3fb34bd-0e60-47f6-9fa6-2bd541ce9deb\") " pod="openshift-network-operator/iptables-alerter-btjpv" May 21 11:30:46.551016 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.550792 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" May 21 11:30:46.551016 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.550785 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e0c35189-f247-439b-9a5e-d199d3963920-etc-sysconfig\") pod \"tuned-lbcnh\" (UID: \"e0c35189-f247-439b-9a5e-d199d3963920\") " pod="openshift-cluster-node-tuning-operator/tuned-lbcnh" May 21 11:30:46.551016 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.550856 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0fd57831-cff3-4f96-8695-cb4d02bc95a9-host\") pod \"node-ca-29k74\" (UID: \"0fd57831-cff3-4f96-8695-cb4d02bc95a9\") " pod="openshift-image-registry/node-ca-29k74" May 21 11:30:46.551016 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.550886 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bfa04605-9b0c-4c9b-a808-598623c60d2d-os-release\") pod \"multus-additional-cni-plugins-9nh44\" (UID: \"bfa04605-9b0c-4c9b-a808-598623c60d2d\") " pod="openshift-multus/multus-additional-cni-plugins-9nh44" May 21 11:30:46.551016 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.550945 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fceead7c-3987-4250-816f-e89182d3b315-system-cni-dir\") pod \"multus-q4k9w\" (UID: \"fceead7c-3987-4250-816f-e89182d3b315\") " pod="openshift-multus/multus-q4k9w" May 21 11:30:46.551016 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.550972 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fceead7c-3987-4250-816f-e89182d3b315-cni-binary-copy\") pod \"multus-q4k9w\" (UID: \"fceead7c-3987-4250-816f-e89182d3b315\") " pod="openshift-multus/multus-q4k9w" May 21 11:30:46.551016 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.551006 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5516274d-6162-4809-bfc0-1f5d4b850794-etc-selinux\") pod \"aws-ebs-csi-driver-node-m2l94\" (UID: \"5516274d-6162-4809-bfc0-1f5d4b850794\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2l94" May 21 11:30:46.551324 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.551034 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e0c35189-f247-439b-9a5e-d199d3963920-etc-sysctl-d\") pod \"tuned-lbcnh\" (UID: \"e0c35189-f247-439b-9a5e-d199d3963920\") " pod="openshift-cluster-node-tuning-operator/tuned-lbcnh" May 21 11:30:46.551324 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.551049 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fceead7c-3987-4250-816f-e89182d3b315-os-release\") pod \"multus-q4k9w\" (UID: \"fceead7c-3987-4250-816f-e89182d3b315\") " pod="openshift-multus/multus-q4k9w" May 21 11:30:46.551324 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.551062 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fceead7c-3987-4250-816f-e89182d3b315-hostroot\") pod \"multus-q4k9w\" (UID: \"fceead7c-3987-4250-816f-e89182d3b315\") " pod="openshift-multus/multus-q4k9w" May 21 11:30:46.551324 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.551105 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5516274d-6162-4809-bfc0-1f5d4b850794-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m2l94\" (UID: \"5516274d-6162-4809-bfc0-1f5d4b850794\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2l94" May 21 11:30:46.551324 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.551143 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5516274d-6162-4809-bfc0-1f5d4b850794-socket-dir\") pod \"aws-ebs-csi-driver-node-m2l94\" (UID: \"5516274d-6162-4809-bfc0-1f5d4b850794\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2l94" May 21 11:30:46.551324 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.551191 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e0c35189-f247-439b-9a5e-d199d3963920-run\") pod \"tuned-lbcnh\" (UID: \"e0c35189-f247-439b-9a5e-d199d3963920\") " pod="openshift-cluster-node-tuning-operator/tuned-lbcnh" May 21 11:30:46.551324 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.551223 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e0c35189-f247-439b-9a5e-d199d3963920-var-lib-kubelet\") pod \"tuned-lbcnh\" (UID: \"e0c35189-f247-439b-9a5e-d199d3963920\") " pod="openshift-cluster-node-tuning-operator/tuned-lbcnh" May 21 11:30:46.551324 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.551259 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e0c35189-f247-439b-9a5e-d199d3963920-etc-tuned\") pod \"tuned-lbcnh\" (UID: \"e0c35189-f247-439b-9a5e-d199d3963920\") " pod="openshift-cluster-node-tuning-operator/tuned-lbcnh" May 21 11:30:46.551324 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.551274 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bfa04605-9b0c-4c9b-a808-598623c60d2d-cnibin\") pod \"multus-additional-cni-plugins-9nh44\" (UID: \"bfa04605-9b0c-4c9b-a808-598623c60d2d\") " pod="openshift-multus/multus-additional-cni-plugins-9nh44" May 21 11:30:46.551324 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.551289 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5516274d-6162-4809-bfc0-1f5d4b850794-registration-dir\") pod \"aws-ebs-csi-driver-node-m2l94\" (UID: \"5516274d-6162-4809-bfc0-1f5d4b850794\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2l94" May 21 11:30:46.551324 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.551309 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5516274d-6162-4809-bfc0-1f5d4b850794-sys-fs\") pod \"aws-ebs-csi-driver-node-m2l94\" (UID: \"5516274d-6162-4809-bfc0-1f5d4b850794\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2l94" May 21 11:30:46.551849 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.551364 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e0c35189-f247-439b-9a5e-d199d3963920-etc-systemd\") pod \"tuned-lbcnh\" (UID: \"e0c35189-f247-439b-9a5e-d199d3963920\") " pod="openshift-cluster-node-tuning-operator/tuned-lbcnh" May 21 11:30:46.551849 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.551415 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bfa04605-9b0c-4c9b-a808-598623c60d2d-system-cni-dir\") pod \"multus-additional-cni-plugins-9nh44\" (UID: \"bfa04605-9b0c-4c9b-a808-598623c60d2d\") " pod="openshift-multus/multus-additional-cni-plugins-9nh44" May 21 11:30:46.551849 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.551440 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bfa04605-9b0c-4c9b-a808-598623c60d2d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9nh44\" (UID: \"bfa04605-9b0c-4c9b-a808-598623c60d2d\") " pod="openshift-multus/multus-additional-cni-plugins-9nh44" May 21 11:30:46.551849 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.551468 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fceead7c-3987-4250-816f-e89182d3b315-host-var-lib-kubelet\") pod \"multus-q4k9w\" (UID: \"fceead7c-3987-4250-816f-e89182d3b315\") " pod="openshift-multus/multus-q4k9w" May 21 11:30:46.551849 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.551527 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fceead7c-3987-4250-816f-e89182d3b315-multus-daemon-config\") pod \"multus-q4k9w\" (UID: \"fceead7c-3987-4250-816f-e89182d3b315\") " pod="openshift-multus/multus-q4k9w" May 21 11:30:46.551849 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.551593 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" May 21 11:30:46.551849 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.551597 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fceead7c-3987-4250-816f-e89182d3b315-host-run-multus-certs\") pod \"multus-q4k9w\" (UID: \"fceead7c-3987-4250-816f-e89182d3b315\") " pod="openshift-multus/multus-q4k9w" May 21 11:30:46.551849 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.551626 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e0c35189-f247-439b-9a5e-d199d3963920-etc-kubernetes\") pod \"tuned-lbcnh\" (UID: \"e0c35189-f247-439b-9a5e-d199d3963920\") " pod="openshift-cluster-node-tuning-operator/tuned-lbcnh" May 21 11:30:46.551849 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.551671 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e0c35189-f247-439b-9a5e-d199d3963920-lib-modules\") pod \"tuned-lbcnh\" (UID: \"e0c35189-f247-439b-9a5e-d199d3963920\") " pod="openshift-cluster-node-tuning-operator/tuned-lbcnh" May 21 11:30:46.551849 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.551695 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59cqf\" (UniqueName: \"kubernetes.io/projected/e0c35189-f247-439b-9a5e-d199d3963920-kube-api-access-59cqf\") pod \"tuned-lbcnh\" (UID: \"e0c35189-f247-439b-9a5e-d199d3963920\") " pod="openshift-cluster-node-tuning-operator/tuned-lbcnh" May 21 11:30:46.551849 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.551719 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fceead7c-3987-4250-816f-e89182d3b315-cnibin\") pod \"multus-q4k9w\" (UID: \"fceead7c-3987-4250-816f-e89182d3b315\") " pod="openshift-multus/multus-q4k9w" May 21 11:30:46.551849 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.551771 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fceead7c-3987-4250-816f-e89182d3b315-host-run-k8s-cni-cncf-io\") pod \"multus-q4k9w\" (UID: \"fceead7c-3987-4250-816f-e89182d3b315\") " pod="openshift-multus/multus-q4k9w" May 21 11:30:46.551849 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.551830 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" May 21 11:30:46.552454 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.551839 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e0c35189-f247-439b-9a5e-d199d3963920-etc-modprobe-d\") pod \"tuned-lbcnh\" (UID: \"e0c35189-f247-439b-9a5e-d199d3963920\") " pod="openshift-cluster-node-tuning-operator/tuned-lbcnh" May 21 11:30:46.552454 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.551881 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e0c35189-f247-439b-9a5e-d199d3963920-etc-sysctl-conf\") pod \"tuned-lbcnh\" (UID: \"e0c35189-f247-439b-9a5e-d199d3963920\") " pod="openshift-cluster-node-tuning-operator/tuned-lbcnh" May 21 11:30:46.552454 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.551899 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e0c35189-f247-439b-9a5e-d199d3963920-sys\") pod \"tuned-lbcnh\" (UID: \"e0c35189-f247-439b-9a5e-d199d3963920\") " pod="openshift-cluster-node-tuning-operator/tuned-lbcnh" May 21 11:30:46.552454 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.551958 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e0c35189-f247-439b-9a5e-d199d3963920-tmp\") pod \"tuned-lbcnh\" (UID: \"e0c35189-f247-439b-9a5e-d199d3963920\") " pod="openshift-cluster-node-tuning-operator/tuned-lbcnh" May 21 11:30:46.552454 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.552014 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bfa04605-9b0c-4c9b-a808-598623c60d2d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9nh44\" (UID: \"bfa04605-9b0c-4c9b-a808-598623c60d2d\") " pod="openshift-multus/multus-additional-cni-plugins-9nh44" May 21 11:30:46.552454 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.552039 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fceead7c-3987-4250-816f-e89182d3b315-host-var-lib-cni-multus\") pod \"multus-q4k9w\" (UID: \"fceead7c-3987-4250-816f-e89182d3b315\") " pod="openshift-multus/multus-q4k9w" May 21 11:30:46.552454 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.552117 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mf8d\" (UniqueName: \"kubernetes.io/projected/fceead7c-3987-4250-816f-e89182d3b315-kube-api-access-4mf8d\") pod \"multus-q4k9w\" (UID: \"fceead7c-3987-4250-816f-e89182d3b315\") " pod="openshift-multus/multus-q4k9w" May 21 11:30:46.552454 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.552167 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b3fb34bd-0e60-47f6-9fa6-2bd541ce9deb-host-slash\") pod \"iptables-alerter-btjpv\" (UID: \"b3fb34bd-0e60-47f6-9fa6-2bd541ce9deb\") " pod="openshift-network-operator/iptables-alerter-btjpv" May 21 11:30:46.552454 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.552181 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5516274d-6162-4809-bfc0-1f5d4b850794-device-dir\") pod \"aws-ebs-csi-driver-node-m2l94\" (UID: \"5516274d-6162-4809-bfc0-1f5d4b850794\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2l94" May 21 11:30:46.553053 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.553023 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xf5gc" May 21 11:30:46.553142 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:46.553119 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xf5gc" podUID="414e12fe-b9ac-4bbf-8470-ad442215f5cb" May 21 11:30:46.553754 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.553666 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" May 21 11:30:46.554107 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.554068 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-vtmmc\"" May 21 11:30:46.554344 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.554328 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" May 21 11:30:46.556111 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.556069 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vw6wd" May 21 11:30:46.556209 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:46.556173 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vw6wd" podUID="8316b46b-bb73-4b4e-805b-da0101c1b222" May 21 11:30:46.559701 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.558986 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-w8vhb" May 21 11:30:46.561936 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.561915 2564 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" May 21 11:30:46.562027 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.561986 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-brz4h\"" May 21 11:30:46.562465 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.562451 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" May 21 11:30:46.562723 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.562710 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" May 21 11:30:46.588207 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.588186 2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-nlxwl" May 21 11:30:46.596288 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.596264 2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-nlxwl" May 21 11:30:46.645227 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.645205 2564 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world" May 21 11:30:46.652346 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.652323 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/45027dd2-2a4d-4ff5-b33f-48e3ee052186-run-ovn\") pod \"ovnkube-node-sr2wt\" (UID: \"45027dd2-2a4d-4ff5-b33f-48e3ee052186\") " pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.652478 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.652352 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/45027dd2-2a4d-4ff5-b33f-48e3ee052186-host-cni-bin\") pod \"ovnkube-node-sr2wt\" (UID: \"45027dd2-2a4d-4ff5-b33f-48e3ee052186\") " pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.652478 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.652386 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdsfn\" (UniqueName: \"kubernetes.io/projected/71722934-df74-4aa7-91e7-9f388d432991-kube-api-access-hdsfn\") pod \"node-resolver-lkfnm\" (UID: \"71722934-df74-4aa7-91e7-9f388d432991\") " pod="openshift-dns/node-resolver-lkfnm" May 21 11:30:46.652478 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.652413 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5516274d-6162-4809-bfc0-1f5d4b850794-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m2l94\" (UID: \"5516274d-6162-4809-bfc0-1f5d4b850794\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2l94" May 21 11:30:46.652478 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.652447 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5516274d-6162-4809-bfc0-1f5d4b850794-socket-dir\") pod \"aws-ebs-csi-driver-node-m2l94\" (UID: \"5516274d-6162-4809-bfc0-1f5d4b850794\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2l94" May 21 11:30:46.652478 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.652463 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e0c35189-f247-439b-9a5e-d199d3963920-run\") pod \"tuned-lbcnh\" (UID: \"e0c35189-f247-439b-9a5e-d199d3963920\") " pod="openshift-cluster-node-tuning-operator/tuned-lbcnh" May 21 11:30:46.652478 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.652478 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e0c35189-f247-439b-9a5e-d199d3963920-etc-tuned\") pod \"tuned-lbcnh\" (UID: \"e0c35189-f247-439b-9a5e-d199d3963920\") " pod="openshift-cluster-node-tuning-operator/tuned-lbcnh" May 21 11:30:46.652736 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.652500 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bfa04605-9b0c-4c9b-a808-598623c60d2d-cnibin\") pod \"multus-additional-cni-plugins-9nh44\" (UID: \"bfa04605-9b0c-4c9b-a808-598623c60d2d\") " pod="openshift-multus/multus-additional-cni-plugins-9nh44" May 21 11:30:46.652736 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.652535 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bfa04605-9b0c-4c9b-a808-598623c60d2d-cnibin\") pod \"multus-additional-cni-plugins-9nh44\" (UID: \"bfa04605-9b0c-4c9b-a808-598623c60d2d\") " pod="openshift-multus/multus-additional-cni-plugins-9nh44" May 21 11:30:46.652736 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.652552 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e0c35189-f247-439b-9a5e-d199d3963920-run\") pod \"tuned-lbcnh\" (UID: \"e0c35189-f247-439b-9a5e-d199d3963920\") " pod="openshift-cluster-node-tuning-operator/tuned-lbcnh" May 21 11:30:46.652736 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.652596 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/45027dd2-2a4d-4ff5-b33f-48e3ee052186-log-socket\") pod \"ovnkube-node-sr2wt\" (UID: \"45027dd2-2a4d-4ff5-b33f-48e3ee052186\") " pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.652736 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.652598 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5516274d-6162-4809-bfc0-1f5d4b850794-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m2l94\" (UID: \"5516274d-6162-4809-bfc0-1f5d4b850794\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2l94" May 21 11:30:46.652736 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.652634 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/71722934-df74-4aa7-91e7-9f388d432991-hosts-file\") pod \"node-resolver-lkfnm\" (UID: \"71722934-df74-4aa7-91e7-9f388d432991\") " pod="openshift-dns/node-resolver-lkfnm" May 21 11:30:46.652736 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.652631 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5516274d-6162-4809-bfc0-1f5d4b850794-socket-dir\") pod \"aws-ebs-csi-driver-node-m2l94\" (UID: \"5516274d-6162-4809-bfc0-1f5d4b850794\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2l94" May 21 11:30:46.652736 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.652667 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5516274d-6162-4809-bfc0-1f5d4b850794-registration-dir\") pod \"aws-ebs-csi-driver-node-m2l94\" (UID: \"5516274d-6162-4809-bfc0-1f5d4b850794\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2l94" May 21 11:30:46.652736 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.652695 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bfa04605-9b0c-4c9b-a808-598623c60d2d-system-cni-dir\") pod \"multus-additional-cni-plugins-9nh44\" (UID: \"bfa04605-9b0c-4c9b-a808-598623c60d2d\") " pod="openshift-multus/multus-additional-cni-plugins-9nh44" May 21 11:30:46.652736 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.652725 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fceead7c-3987-4250-816f-e89182d3b315-host-var-lib-kubelet\") pod \"multus-q4k9w\" (UID: \"fceead7c-3987-4250-816f-e89182d3b315\") " pod="openshift-multus/multus-q4k9w" May 21 11:30:46.652736 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.652732 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5516274d-6162-4809-bfc0-1f5d4b850794-registration-dir\") pod \"aws-ebs-csi-driver-node-m2l94\" (UID: \"5516274d-6162-4809-bfc0-1f5d4b850794\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2l94" May 21 11:30:46.653184 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.652748 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fceead7c-3987-4250-816f-e89182d3b315-host-run-multus-certs\") pod \"multus-q4k9w\" (UID: \"fceead7c-3987-4250-816f-e89182d3b315\") " pod="openshift-multus/multus-q4k9w" May 21 11:30:46.653184 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.652758 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bfa04605-9b0c-4c9b-a808-598623c60d2d-system-cni-dir\") pod \"multus-additional-cni-plugins-9nh44\" (UID: \"bfa04605-9b0c-4c9b-a808-598623c60d2d\") " pod="openshift-multus/multus-additional-cni-plugins-9nh44" May 21 11:30:46.653184 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.652772 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fceead7c-3987-4250-816f-e89182d3b315-host-var-lib-kubelet\") pod \"multus-q4k9w\" (UID: \"fceead7c-3987-4250-816f-e89182d3b315\") " pod="openshift-multus/multus-q4k9w" May 21 11:30:46.653184 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.652776 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/45027dd2-2a4d-4ff5-b33f-48e3ee052186-systemd-units\") pod \"ovnkube-node-sr2wt\" (UID: \"45027dd2-2a4d-4ff5-b33f-48e3ee052186\") " pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.653184 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.652804 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqt78\" (UniqueName: \"kubernetes.io/projected/414e12fe-b9ac-4bbf-8470-ad442215f5cb-kube-api-access-cqt78\") pod \"network-metrics-daemon-xf5gc\" (UID: \"414e12fe-b9ac-4bbf-8470-ad442215f5cb\") " pod="openshift-multus/network-metrics-daemon-xf5gc" May 21 11:30:46.653184 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.652810 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fceead7c-3987-4250-816f-e89182d3b315-host-run-multus-certs\") pod \"multus-q4k9w\" (UID: \"fceead7c-3987-4250-816f-e89182d3b315\") " pod="openshift-multus/multus-q4k9w" May 21 11:30:46.653184 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.652829 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e0c35189-f247-439b-9a5e-d199d3963920-etc-kubernetes\") pod \"tuned-lbcnh\" (UID: \"e0c35189-f247-439b-9a5e-d199d3963920\") " pod="openshift-cluster-node-tuning-operator/tuned-lbcnh" May 21 11:30:46.653184 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.652855 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e0c35189-f247-439b-9a5e-d199d3963920-lib-modules\") pod \"tuned-lbcnh\" (UID: \"e0c35189-f247-439b-9a5e-d199d3963920\") " pod="openshift-cluster-node-tuning-operator/tuned-lbcnh" May 21 11:30:46.653184 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.652897 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-59cqf\" (UniqueName: \"kubernetes.io/projected/e0c35189-f247-439b-9a5e-d199d3963920-kube-api-access-59cqf\") pod \"tuned-lbcnh\" (UID: \"e0c35189-f247-439b-9a5e-d199d3963920\") " pod="openshift-cluster-node-tuning-operator/tuned-lbcnh" May 21 11:30:46.653184 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.652909 2564 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" May 21 11:30:46.653184 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.652923 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fceead7c-3987-4250-816f-e89182d3b315-host-run-k8s-cni-cncf-io\") pod \"multus-q4k9w\" (UID: \"fceead7c-3987-4250-816f-e89182d3b315\") " pod="openshift-multus/multus-q4k9w" May 21 11:30:46.653184 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.652947 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e0c35189-f247-439b-9a5e-d199d3963920-etc-modprobe-d\") pod \"tuned-lbcnh\" (UID: \"e0c35189-f247-439b-9a5e-d199d3963920\") " pod="openshift-cluster-node-tuning-operator/tuned-lbcnh" May 21 11:30:46.653184 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.652971 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e0c35189-f247-439b-9a5e-d199d3963920-sys\") pod \"tuned-lbcnh\" (UID: \"e0c35189-f247-439b-9a5e-d199d3963920\") " pod="openshift-cluster-node-tuning-operator/tuned-lbcnh" May 21 11:30:46.653184 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.652945 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e0c35189-f247-439b-9a5e-d199d3963920-etc-kubernetes\") pod \"tuned-lbcnh\" (UID: \"e0c35189-f247-439b-9a5e-d199d3963920\") " pod="openshift-cluster-node-tuning-operator/tuned-lbcnh" May 21 11:30:46.653184 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.652994 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fceead7c-3987-4250-816f-e89182d3b315-host-var-lib-cni-multus\") pod \"multus-q4k9w\" (UID: \"fceead7c-3987-4250-816f-e89182d3b315\") " pod="openshift-multus/multus-q4k9w" May 21 11:30:46.653184 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.652998 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e0c35189-f247-439b-9a5e-d199d3963920-lib-modules\") pod \"tuned-lbcnh\" (UID: \"e0c35189-f247-439b-9a5e-d199d3963920\") " pod="openshift-cluster-node-tuning-operator/tuned-lbcnh" May 21 11:30:46.653184 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.653017 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4mf8d\" (UniqueName: \"kubernetes.io/projected/fceead7c-3987-4250-816f-e89182d3b315-kube-api-access-4mf8d\") pod \"multus-q4k9w\" (UID: \"fceead7c-3987-4250-816f-e89182d3b315\") " pod="openshift-multus/multus-q4k9w" May 21 11:30:46.653184 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.653034 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fceead7c-3987-4250-816f-e89182d3b315-host-run-k8s-cni-cncf-io\") pod \"multus-q4k9w\" (UID: \"fceead7c-3987-4250-816f-e89182d3b315\") " pod="openshift-multus/multus-q4k9w" May 21 11:30:46.654041 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.653039 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/45027dd2-2a4d-4ff5-b33f-48e3ee052186-host-cni-netd\") pod \"ovnkube-node-sr2wt\" (UID: \"45027dd2-2a4d-4ff5-b33f-48e3ee052186\") " pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.654041 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.653052 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e0c35189-f247-439b-9a5e-d199d3963920-sys\") pod \"tuned-lbcnh\" (UID: \"e0c35189-f247-439b-9a5e-d199d3963920\") " pod="openshift-cluster-node-tuning-operator/tuned-lbcnh" May 21 11:30:46.654041 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.653067 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b3fb34bd-0e60-47f6-9fa6-2bd541ce9deb-host-slash\") pod \"iptables-alerter-btjpv\" (UID: \"b3fb34bd-0e60-47f6-9fa6-2bd541ce9deb\") " pod="openshift-network-operator/iptables-alerter-btjpv" May 21 11:30:46.654041 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.653100 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fceead7c-3987-4250-816f-e89182d3b315-host-var-lib-cni-multus\") pod \"multus-q4k9w\" (UID: \"fceead7c-3987-4250-816f-e89182d3b315\") " pod="openshift-multus/multus-q4k9w" May 21 11:30:46.654041 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.653067 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e0c35189-f247-439b-9a5e-d199d3963920-etc-modprobe-d\") pod \"tuned-lbcnh\" (UID: \"e0c35189-f247-439b-9a5e-d199d3963920\") " pod="openshift-cluster-node-tuning-operator/tuned-lbcnh" May 21 11:30:46.654041 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.653114 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bfa04605-9b0c-4c9b-a808-598623c60d2d-cni-binary-copy\") pod \"multus-additional-cni-plugins-9nh44\" (UID: \"bfa04605-9b0c-4c9b-a808-598623c60d2d\") " pod="openshift-multus/multus-additional-cni-plugins-9nh44" May 21 11:30:46.654041 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.653157 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fceead7c-3987-4250-816f-e89182d3b315-multus-socket-dir-parent\") pod \"multus-q4k9w\" (UID: \"fceead7c-3987-4250-816f-e89182d3b315\") " pod="openshift-multus/multus-q4k9w" May 21 11:30:46.654041 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.653163 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b3fb34bd-0e60-47f6-9fa6-2bd541ce9deb-host-slash\") pod \"iptables-alerter-btjpv\" (UID: \"b3fb34bd-0e60-47f6-9fa6-2bd541ce9deb\") " pod="openshift-network-operator/iptables-alerter-btjpv" May 21 11:30:46.654041 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.653193 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45027dd2-2a4d-4ff5-b33f-48e3ee052186-run-openvswitch\") pod \"ovnkube-node-sr2wt\" (UID: \"45027dd2-2a4d-4ff5-b33f-48e3ee052186\") " pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.654041 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.653207 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fceead7c-3987-4250-816f-e89182d3b315-multus-socket-dir-parent\") pod \"multus-q4k9w\" (UID: \"fceead7c-3987-4250-816f-e89182d3b315\") " pod="openshift-multus/multus-q4k9w" May 21 11:30:46.654041 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.653317 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hfxxv\" (UniqueName: \"kubernetes.io/projected/b3fb34bd-0e60-47f6-9fa6-2bd541ce9deb-kube-api-access-hfxxv\") pod \"iptables-alerter-btjpv\" (UID: \"b3fb34bd-0e60-47f6-9fa6-2bd541ce9deb\") " pod="openshift-network-operator/iptables-alerter-btjpv" May 21 11:30:46.654041 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.653336 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-x7qs9\" (UniqueName: \"kubernetes.io/projected/5516274d-6162-4809-bfc0-1f5d4b850794-kube-api-access-x7qs9\") pod \"aws-ebs-csi-driver-node-m2l94\" (UID: \"5516274d-6162-4809-bfc0-1f5d4b850794\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2l94" May 21 11:30:46.654041 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.653350 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e0c35189-f247-439b-9a5e-d199d3963920-host\") pod \"tuned-lbcnh\" (UID: \"e0c35189-f247-439b-9a5e-d199d3963920\") " pod="openshift-cluster-node-tuning-operator/tuned-lbcnh" May 21 11:30:46.654041 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.653369 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fceead7c-3987-4250-816f-e89182d3b315-host-run-netns\") pod \"multus-q4k9w\" (UID: \"fceead7c-3987-4250-816f-e89182d3b315\") " pod="openshift-multus/multus-q4k9w" May 21 11:30:46.654041 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.653407 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fceead7c-3987-4250-816f-e89182d3b315-host-run-netns\") pod \"multus-q4k9w\" (UID: \"fceead7c-3987-4250-816f-e89182d3b315\") " pod="openshift-multus/multus-q4k9w" May 21 11:30:46.654041 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.653425 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e0c35189-f247-439b-9a5e-d199d3963920-host\") pod \"tuned-lbcnh\" (UID: \"e0c35189-f247-439b-9a5e-d199d3963920\") " pod="openshift-cluster-node-tuning-operator/tuned-lbcnh" May 21 11:30:46.654041 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.653497 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/45027dd2-2a4d-4ff5-b33f-48e3ee052186-node-log\") pod \"ovnkube-node-sr2wt\" (UID: \"45027dd2-2a4d-4ff5-b33f-48e3ee052186\") " pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.654768 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.653536 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45027dd2-2a4d-4ff5-b33f-48e3ee052186-host-run-ovn-kubernetes\") pod \"ovnkube-node-sr2wt\" (UID: \"45027dd2-2a4d-4ff5-b33f-48e3ee052186\") " pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.654768 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.653567 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/45027dd2-2a4d-4ff5-b33f-48e3ee052186-ovnkube-script-lib\") pod \"ovnkube-node-sr2wt\" (UID: \"45027dd2-2a4d-4ff5-b33f-48e3ee052186\") " pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.654768 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.653593 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b3fb34bd-0e60-47f6-9fa6-2bd541ce9deb-iptables-alerter-script\") pod \"iptables-alerter-btjpv\" (UID: \"b3fb34bd-0e60-47f6-9fa6-2bd541ce9deb\") " pod="openshift-network-operator/iptables-alerter-btjpv" May 21 11:30:46.654768 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.653632 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e0c35189-f247-439b-9a5e-d199d3963920-etc-sysconfig\") pod \"tuned-lbcnh\" (UID: \"e0c35189-f247-439b-9a5e-d199d3963920\") " pod="openshift-cluster-node-tuning-operator/tuned-lbcnh" May 21 11:30:46.654768 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.653665 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0fd57831-cff3-4f96-8695-cb4d02bc95a9-host\") pod \"node-ca-29k74\" (UID: \"0fd57831-cff3-4f96-8695-cb4d02bc95a9\") " pod="openshift-image-registry/node-ca-29k74" May 21 11:30:46.654768 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.653679 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e0c35189-f247-439b-9a5e-d199d3963920-etc-sysconfig\") pod \"tuned-lbcnh\" (UID: \"e0c35189-f247-439b-9a5e-d199d3963920\") " pod="openshift-cluster-node-tuning-operator/tuned-lbcnh" May 21 11:30:46.654768 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.653690 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bfa04605-9b0c-4c9b-a808-598623c60d2d-os-release\") pod \"multus-additional-cni-plugins-9nh44\" (UID: \"bfa04605-9b0c-4c9b-a808-598623c60d2d\") " pod="openshift-multus/multus-additional-cni-plugins-9nh44" May 21 11:30:46.654768 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.653721 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/45027dd2-2a4d-4ff5-b33f-48e3ee052186-host-kubelet\") pod \"ovnkube-node-sr2wt\" (UID: \"45027dd2-2a4d-4ff5-b33f-48e3ee052186\") " pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.654768 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.653751 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71722934-df74-4aa7-91e7-9f388d432991-tmp-dir\") pod \"node-resolver-lkfnm\" (UID: \"71722934-df74-4aa7-91e7-9f388d432991\") " pod="openshift-dns/node-resolver-lkfnm" May 21 11:30:46.654768 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.653724 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0fd57831-cff3-4f96-8695-cb4d02bc95a9-host\") pod \"node-ca-29k74\" (UID: \"0fd57831-cff3-4f96-8695-cb4d02bc95a9\") " pod="openshift-image-registry/node-ca-29k74" May 21 11:30:46.654768 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.653764 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bfa04605-9b0c-4c9b-a808-598623c60d2d-os-release\") pod \"multus-additional-cni-plugins-9nh44\" (UID: \"bfa04605-9b0c-4c9b-a808-598623c60d2d\") " pod="openshift-multus/multus-additional-cni-plugins-9nh44" May 21 11:30:46.654768 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.653803 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5516274d-6162-4809-bfc0-1f5d4b850794-etc-selinux\") pod \"aws-ebs-csi-driver-node-m2l94\" (UID: \"5516274d-6162-4809-bfc0-1f5d4b850794\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2l94" May 21 11:30:46.654768 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.653830 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fceead7c-3987-4250-816f-e89182d3b315-hostroot\") pod \"multus-q4k9w\" (UID: \"fceead7c-3987-4250-816f-e89182d3b315\") " pod="openshift-multus/multus-q4k9w" May 21 11:30:46.654768 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.653855 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7n49\" (UniqueName: \"kubernetes.io/projected/45027dd2-2a4d-4ff5-b33f-48e3ee052186-kube-api-access-r7n49\") pod \"ovnkube-node-sr2wt\" (UID: \"45027dd2-2a4d-4ff5-b33f-48e3ee052186\") " pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.654768 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.653879 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fceead7c-3987-4250-816f-e89182d3b315-hostroot\") pod \"multus-q4k9w\" (UID: \"fceead7c-3987-4250-816f-e89182d3b315\") " pod="openshift-multus/multus-q4k9w" May 21 11:30:46.654768 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.653883 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e0c35189-f247-439b-9a5e-d199d3963920-var-lib-kubelet\") pod \"tuned-lbcnh\" (UID: \"e0c35189-f247-439b-9a5e-d199d3963920\") " pod="openshift-cluster-node-tuning-operator/tuned-lbcnh" May 21 11:30:46.654768 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.653908 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45027dd2-2a4d-4ff5-b33f-48e3ee052186-var-lib-openvswitch\") pod \"ovnkube-node-sr2wt\" (UID: \"45027dd2-2a4d-4ff5-b33f-48e3ee052186\") " pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.655759 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.653912 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5516274d-6162-4809-bfc0-1f5d4b850794-etc-selinux\") pod \"aws-ebs-csi-driver-node-m2l94\" (UID: \"5516274d-6162-4809-bfc0-1f5d4b850794\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2l94" May 21 11:30:46.655759 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.653935 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/414e12fe-b9ac-4bbf-8470-ad442215f5cb-metrics-certs\") pod \"network-metrics-daemon-xf5gc\" (UID: \"414e12fe-b9ac-4bbf-8470-ad442215f5cb\") " pod="openshift-multus/network-metrics-daemon-xf5gc" May 21 11:30:46.655759 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.653976 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e0c35189-f247-439b-9a5e-d199d3963920-var-lib-kubelet\") pod \"tuned-lbcnh\" (UID: \"e0c35189-f247-439b-9a5e-d199d3963920\") " pod="openshift-cluster-node-tuning-operator/tuned-lbcnh" May 21 11:30:46.655759 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.653979 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvd92\" (UniqueName: \"kubernetes.io/projected/8316b46b-bb73-4b4e-805b-da0101c1b222-kube-api-access-kvd92\") pod \"network-check-target-vw6wd\" (UID: \"8316b46b-bb73-4b4e-805b-da0101c1b222\") " pod="openshift-network-diagnostics/network-check-target-vw6wd" May 21 11:30:46.655759 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.654022 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/449c0696-add7-49fb-9892-5c100cdc6d7a-konnectivity-ca\") pod \"konnectivity-agent-w8vhb\" (UID: \"449c0696-add7-49fb-9892-5c100cdc6d7a\") " pod="kube-system/konnectivity-agent-w8vhb" May 21 11:30:46.655759 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.654052 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b3fb34bd-0e60-47f6-9fa6-2bd541ce9deb-iptables-alerter-script\") pod \"iptables-alerter-btjpv\" (UID: \"b3fb34bd-0e60-47f6-9fa6-2bd541ce9deb\") " pod="openshift-network-operator/iptables-alerter-btjpv" May 21 11:30:46.655759 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.654052 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5516274d-6162-4809-bfc0-1f5d4b850794-sys-fs\") pod \"aws-ebs-csi-driver-node-m2l94\" (UID: \"5516274d-6162-4809-bfc0-1f5d4b850794\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2l94" May 21 11:30:46.655759 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.654131 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e0c35189-f247-439b-9a5e-d199d3963920-etc-systemd\") pod \"tuned-lbcnh\" (UID: \"e0c35189-f247-439b-9a5e-d199d3963920\") " pod="openshift-cluster-node-tuning-operator/tuned-lbcnh" May 21 11:30:46.655759 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.654149 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bfa04605-9b0c-4c9b-a808-598623c60d2d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9nh44\" (UID: \"bfa04605-9b0c-4c9b-a808-598623c60d2d\") " pod="openshift-multus/multus-additional-cni-plugins-9nh44" May 21 11:30:46.655759 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.654128 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5516274d-6162-4809-bfc0-1f5d4b850794-sys-fs\") pod \"aws-ebs-csi-driver-node-m2l94\" (UID: \"5516274d-6162-4809-bfc0-1f5d4b850794\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2l94" May 21 11:30:46.655759 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.654168 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fceead7c-3987-4250-816f-e89182d3b315-multus-daemon-config\") pod \"multus-q4k9w\" (UID: \"fceead7c-3987-4250-816f-e89182d3b315\") " pod="openshift-multus/multus-q4k9w" May 21 11:30:46.655759 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.654202 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45027dd2-2a4d-4ff5-b33f-48e3ee052186-etc-openvswitch\") pod \"ovnkube-node-sr2wt\" (UID: \"45027dd2-2a4d-4ff5-b33f-48e3ee052186\") " pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.655759 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.654221 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e0c35189-f247-439b-9a5e-d199d3963920-etc-systemd\") pod \"tuned-lbcnh\" (UID: \"e0c35189-f247-439b-9a5e-d199d3963920\") " pod="openshift-cluster-node-tuning-operator/tuned-lbcnh" May 21 11:30:46.655759 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.654254 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/449c0696-add7-49fb-9892-5c100cdc6d7a-agent-certs\") pod \"konnectivity-agent-w8vhb\" (UID: \"449c0696-add7-49fb-9892-5c100cdc6d7a\") " pod="kube-system/konnectivity-agent-w8vhb" May 21 11:30:46.655759 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.654274 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bfa04605-9b0c-4c9b-a808-598623c60d2d-cni-binary-copy\") pod \"multus-additional-cni-plugins-9nh44\" (UID: \"bfa04605-9b0c-4c9b-a808-598623c60d2d\") " pod="openshift-multus/multus-additional-cni-plugins-9nh44" May 21 11:30:46.655759 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.654295 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fceead7c-3987-4250-816f-e89182d3b315-cnibin\") pod \"multus-q4k9w\" (UID: \"fceead7c-3987-4250-816f-e89182d3b315\") " pod="openshift-multus/multus-q4k9w" May 21 11:30:46.655759 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.654306 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bfa04605-9b0c-4c9b-a808-598623c60d2d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9nh44\" (UID: \"bfa04605-9b0c-4c9b-a808-598623c60d2d\") " pod="openshift-multus/multus-additional-cni-plugins-9nh44" May 21 11:30:46.656568 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.654327 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/45027dd2-2a4d-4ff5-b33f-48e3ee052186-host-run-netns\") pod \"ovnkube-node-sr2wt\" (UID: \"45027dd2-2a4d-4ff5-b33f-48e3ee052186\") " pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.656568 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.654354 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/45027dd2-2a4d-4ff5-b33f-48e3ee052186-run-systemd\") pod \"ovnkube-node-sr2wt\" (UID: \"45027dd2-2a4d-4ff5-b33f-48e3ee052186\") " pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.656568 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.654371 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/45027dd2-2a4d-4ff5-b33f-48e3ee052186-ovnkube-config\") pod \"ovnkube-node-sr2wt\" (UID: \"45027dd2-2a4d-4ff5-b33f-48e3ee052186\") " pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.656568 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.654373 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fceead7c-3987-4250-816f-e89182d3b315-cnibin\") pod \"multus-q4k9w\" (UID: \"fceead7c-3987-4250-816f-e89182d3b315\") " pod="openshift-multus/multus-q4k9w" May 21 11:30:46.656568 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.654390 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e0c35189-f247-439b-9a5e-d199d3963920-etc-sysctl-conf\") pod \"tuned-lbcnh\" (UID: \"e0c35189-f247-439b-9a5e-d199d3963920\") " pod="openshift-cluster-node-tuning-operator/tuned-lbcnh" May 21 11:30:46.656568 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.654422 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e0c35189-f247-439b-9a5e-d199d3963920-tmp\") pod \"tuned-lbcnh\" (UID: \"e0c35189-f247-439b-9a5e-d199d3963920\") " pod="openshift-cluster-node-tuning-operator/tuned-lbcnh" May 21 11:30:46.656568 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.654438 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bfa04605-9b0c-4c9b-a808-598623c60d2d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9nh44\" (UID: \"bfa04605-9b0c-4c9b-a808-598623c60d2d\") " pod="openshift-multus/multus-additional-cni-plugins-9nh44" May 21 11:30:46.656568 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.654460 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5516274d-6162-4809-bfc0-1f5d4b850794-device-dir\") pod \"aws-ebs-csi-driver-node-m2l94\" (UID: \"5516274d-6162-4809-bfc0-1f5d4b850794\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2l94" May 21 11:30:46.656568 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.654484 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0fd57831-cff3-4f96-8695-cb4d02bc95a9-serviceca\") pod \"node-ca-29k74\" (UID: \"0fd57831-cff3-4f96-8695-cb4d02bc95a9\") " pod="openshift-image-registry/node-ca-29k74" May 21 11:30:46.656568 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.654508 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bfa04605-9b0c-4c9b-a808-598623c60d2d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9nh44\" (UID: \"bfa04605-9b0c-4c9b-a808-598623c60d2d\") " pod="openshift-multus/multus-additional-cni-plugins-9nh44" May 21 11:30:46.656568 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.654530 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5516274d-6162-4809-bfc0-1f5d4b850794-device-dir\") pod \"aws-ebs-csi-driver-node-m2l94\" (UID: \"5516274d-6162-4809-bfc0-1f5d4b850794\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2l94" May 21 11:30:46.656568 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.654535 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4bplk\" (UniqueName: \"kubernetes.io/projected/bfa04605-9b0c-4c9b-a808-598623c60d2d-kube-api-access-4bplk\") pod \"multus-additional-cni-plugins-9nh44\" (UID: \"bfa04605-9b0c-4c9b-a808-598623c60d2d\") " pod="openshift-multus/multus-additional-cni-plugins-9nh44" May 21 11:30:46.656568 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.654551 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e0c35189-f247-439b-9a5e-d199d3963920-etc-sysctl-conf\") pod \"tuned-lbcnh\" (UID: \"e0c35189-f247-439b-9a5e-d199d3963920\") " pod="openshift-cluster-node-tuning-operator/tuned-lbcnh" May 21 11:30:46.656568 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.654596 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fceead7c-3987-4250-816f-e89182d3b315-multus-cni-dir\") pod \"multus-q4k9w\" (UID: \"fceead7c-3987-4250-816f-e89182d3b315\") " pod="openshift-multus/multus-q4k9w" May 21 11:30:46.656568 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.654657 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fceead7c-3987-4250-816f-e89182d3b315-multus-cni-dir\") pod \"multus-q4k9w\" (UID: \"fceead7c-3987-4250-816f-e89182d3b315\") " pod="openshift-multus/multus-q4k9w" May 21 11:30:46.656568 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.654681 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fceead7c-3987-4250-816f-e89182d3b315-multus-daemon-config\") pod \"multus-q4k9w\" (UID: \"fceead7c-3987-4250-816f-e89182d3b315\") " pod="openshift-multus/multus-q4k9w" May 21 11:30:46.656568 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.654689 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fceead7c-3987-4250-816f-e89182d3b315-host-var-lib-cni-bin\") pod \"multus-q4k9w\" (UID: \"fceead7c-3987-4250-816f-e89182d3b315\") " pod="openshift-multus/multus-q4k9w" May 21 11:30:46.657059 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.654725 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fceead7c-3987-4250-816f-e89182d3b315-etc-kubernetes\") pod \"multus-q4k9w\" (UID: \"fceead7c-3987-4250-816f-e89182d3b315\") " pod="openshift-multus/multus-q4k9w" May 21 11:30:46.657059 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.654728 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fceead7c-3987-4250-816f-e89182d3b315-host-var-lib-cni-bin\") pod \"multus-q4k9w\" (UID: \"fceead7c-3987-4250-816f-e89182d3b315\") " pod="openshift-multus/multus-q4k9w" May 21 11:30:46.657059 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.654759 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/45027dd2-2a4d-4ff5-b33f-48e3ee052186-host-slash\") pod \"ovnkube-node-sr2wt\" (UID: \"45027dd2-2a4d-4ff5-b33f-48e3ee052186\") " pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.657059 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.654767 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fceead7c-3987-4250-816f-e89182d3b315-etc-kubernetes\") pod \"multus-q4k9w\" (UID: \"fceead7c-3987-4250-816f-e89182d3b315\") " pod="openshift-multus/multus-q4k9w" May 21 11:30:46.657059 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.654786 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kmkdw\" (UniqueName: \"kubernetes.io/projected/0fd57831-cff3-4f96-8695-cb4d02bc95a9-kube-api-access-kmkdw\") pod \"node-ca-29k74\" (UID: \"0fd57831-cff3-4f96-8695-cb4d02bc95a9\") " pod="openshift-image-registry/node-ca-29k74" May 21 11:30:46.657059 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.654808 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fceead7c-3987-4250-816f-e89182d3b315-multus-conf-dir\") pod \"multus-q4k9w\" (UID: \"fceead7c-3987-4250-816f-e89182d3b315\") " pod="openshift-multus/multus-q4k9w" May 21 11:30:46.657059 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.654826 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/45027dd2-2a4d-4ff5-b33f-48e3ee052186-env-overrides\") pod \"ovnkube-node-sr2wt\" (UID: \"45027dd2-2a4d-4ff5-b33f-48e3ee052186\") " pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.657059 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.654850 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fceead7c-3987-4250-816f-e89182d3b315-system-cni-dir\") pod \"multus-q4k9w\" (UID: \"fceead7c-3987-4250-816f-e89182d3b315\") " pod="openshift-multus/multus-q4k9w" May 21 11:30:46.657059 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.654877 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fceead7c-3987-4250-816f-e89182d3b315-cni-binary-copy\") pod \"multus-q4k9w\" (UID: \"fceead7c-3987-4250-816f-e89182d3b315\") " pod="openshift-multus/multus-q4k9w" May 21 11:30:46.657059 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.654892 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fceead7c-3987-4250-816f-e89182d3b315-multus-conf-dir\") pod \"multus-q4k9w\" (UID: \"fceead7c-3987-4250-816f-e89182d3b315\") " pod="openshift-multus/multus-q4k9w" May 21 11:30:46.657059 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.654906 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bfa04605-9b0c-4c9b-a808-598623c60d2d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9nh44\" (UID: \"bfa04605-9b0c-4c9b-a808-598623c60d2d\") " pod="openshift-multus/multus-additional-cni-plugins-9nh44" May 21 11:30:46.657059 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.654905 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45027dd2-2a4d-4ff5-b33f-48e3ee052186-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sr2wt\" (UID: \"45027dd2-2a4d-4ff5-b33f-48e3ee052186\") " pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.657059 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.655042 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fceead7c-3987-4250-816f-e89182d3b315-system-cni-dir\") pod \"multus-q4k9w\" (UID: \"fceead7c-3987-4250-816f-e89182d3b315\") " pod="openshift-multus/multus-q4k9w" May 21 11:30:46.657059 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.655101 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/45027dd2-2a4d-4ff5-b33f-48e3ee052186-ovn-node-metrics-cert\") pod \"ovnkube-node-sr2wt\" (UID: \"45027dd2-2a4d-4ff5-b33f-48e3ee052186\") " pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.657059 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.655154 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bfa04605-9b0c-4c9b-a808-598623c60d2d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9nh44\" (UID: \"bfa04605-9b0c-4c9b-a808-598623c60d2d\") " pod="openshift-multus/multus-additional-cni-plugins-9nh44" May 21 11:30:46.657059 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.655165 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0fd57831-cff3-4f96-8695-cb4d02bc95a9-serviceca\") pod \"node-ca-29k74\" (UID: \"0fd57831-cff3-4f96-8695-cb4d02bc95a9\") " pod="openshift-image-registry/node-ca-29k74" May 21 11:30:46.657059 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.655207 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e0c35189-f247-439b-9a5e-d199d3963920-etc-sysctl-d\") pod \"tuned-lbcnh\" (UID: \"e0c35189-f247-439b-9a5e-d199d3963920\") " pod="openshift-cluster-node-tuning-operator/tuned-lbcnh" May 21 11:30:46.657561 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.655229 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fceead7c-3987-4250-816f-e89182d3b315-os-release\") pod \"multus-q4k9w\" (UID: \"fceead7c-3987-4250-816f-e89182d3b315\") " pod="openshift-multus/multus-q4k9w" May 21 11:30:46.657561 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.655335 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fceead7c-3987-4250-816f-e89182d3b315-os-release\") pod \"multus-q4k9w\" (UID: \"fceead7c-3987-4250-816f-e89182d3b315\") " pod="openshift-multus/multus-q4k9w" May 21 11:30:46.657561 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.655359 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e0c35189-f247-439b-9a5e-d199d3963920-etc-sysctl-d\") pod \"tuned-lbcnh\" (UID: \"e0c35189-f247-439b-9a5e-d199d3963920\") " pod="openshift-cluster-node-tuning-operator/tuned-lbcnh" May 21 11:30:46.657561 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.655437 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fceead7c-3987-4250-816f-e89182d3b315-cni-binary-copy\") pod \"multus-q4k9w\" (UID: \"fceead7c-3987-4250-816f-e89182d3b315\") " pod="openshift-multus/multus-q4k9w" May 21 11:30:46.657561 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.656010 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e0c35189-f247-439b-9a5e-d199d3963920-etc-tuned\") pod \"tuned-lbcnh\" (UID: \"e0c35189-f247-439b-9a5e-d199d3963920\") " pod="openshift-cluster-node-tuning-operator/tuned-lbcnh" May 21 11:30:46.657561 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.656757 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e0c35189-f247-439b-9a5e-d199d3963920-tmp\") pod \"tuned-lbcnh\" (UID: \"e0c35189-f247-439b-9a5e-d199d3963920\") " pod="openshift-cluster-node-tuning-operator/tuned-lbcnh" May 21 11:30:46.675889 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.675865 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-59cqf\" (UniqueName: \"kubernetes.io/projected/e0c35189-f247-439b-9a5e-d199d3963920-kube-api-access-59cqf\") pod \"tuned-lbcnh\" (UID: \"e0c35189-f247-439b-9a5e-d199d3963920\") " pod="openshift-cluster-node-tuning-operator/tuned-lbcnh" May 21 11:30:46.675889 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.675874 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7qs9\" (UniqueName: \"kubernetes.io/projected/5516274d-6162-4809-bfc0-1f5d4b850794-kube-api-access-x7qs9\") pod \"aws-ebs-csi-driver-node-m2l94\" (UID: \"5516274d-6162-4809-bfc0-1f5d4b850794\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2l94" May 21 11:30:46.676060 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.676010 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmkdw\" (UniqueName: \"kubernetes.io/projected/0fd57831-cff3-4f96-8695-cb4d02bc95a9-kube-api-access-kmkdw\") pod \"node-ca-29k74\" (UID: \"0fd57831-cff3-4f96-8695-cb4d02bc95a9\") " pod="openshift-image-registry/node-ca-29k74" May 21 11:30:46.677425 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.677403 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfxxv\" (UniqueName: \"kubernetes.io/projected/b3fb34bd-0e60-47f6-9fa6-2bd541ce9deb-kube-api-access-hfxxv\") pod \"iptables-alerter-btjpv\" (UID: \"b3fb34bd-0e60-47f6-9fa6-2bd541ce9deb\") " pod="openshift-network-operator/iptables-alerter-btjpv" May 21 11:30:46.678404 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.678387 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bplk\" (UniqueName: \"kubernetes.io/projected/bfa04605-9b0c-4c9b-a808-598623c60d2d-kube-api-access-4bplk\") pod \"multus-additional-cni-plugins-9nh44\" (UID: \"bfa04605-9b0c-4c9b-a808-598623c60d2d\") " pod="openshift-multus/multus-additional-cni-plugins-9nh44" May 21 11:30:46.678649 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.678632 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mf8d\" (UniqueName: \"kubernetes.io/projected/fceead7c-3987-4250-816f-e89182d3b315-kube-api-access-4mf8d\") pod \"multus-q4k9w\" (UID: \"fceead7c-3987-4250-816f-e89182d3b315\") " pod="openshift-multus/multus-q4k9w" May 21 11:30:46.742360 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:46.742319 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0694ef6a59bf20f80056d0d4f3f2967d.slice/crio-6bfb782524b2eb154b63735c8c97516b2bcc4a01d7ce9c0290f8fa91882b8ea7 WatchSource:0}: Error finding container 6bfb782524b2eb154b63735c8c97516b2bcc4a01d7ce9c0290f8fa91882b8ea7: Status 404 returned error can't find the container with id 6bfb782524b2eb154b63735c8c97516b2bcc4a01d7ce9c0290f8fa91882b8ea7 May 21 11:30:46.747190 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.747171 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 21 11:30:46.751871 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:46.751846 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3da4ba4cdfe6fc200fbc8989e0319648.slice/crio-a87c6ae30ffe1998927010c65f3373f717d1cd7fcdf63647dc7b1b84cd1c482c WatchSource:0}: Error finding container a87c6ae30ffe1998927010c65f3373f717d1cd7fcdf63647dc7b1b84cd1c482c: Status 404 returned error can't find the container with id a87c6ae30ffe1998927010c65f3373f717d1cd7fcdf63647dc7b1b84cd1c482c May 21 11:30:46.755711 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.755691 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/45027dd2-2a4d-4ff5-b33f-48e3ee052186-systemd-units\") pod \"ovnkube-node-sr2wt\" (UID: \"45027dd2-2a4d-4ff5-b33f-48e3ee052186\") " pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.755789 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.755719 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cqt78\" (UniqueName: \"kubernetes.io/projected/414e12fe-b9ac-4bbf-8470-ad442215f5cb-kube-api-access-cqt78\") pod \"network-metrics-daemon-xf5gc\" (UID: \"414e12fe-b9ac-4bbf-8470-ad442215f5cb\") " pod="openshift-multus/network-metrics-daemon-xf5gc" May 21 11:30:46.755789 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.755738 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/45027dd2-2a4d-4ff5-b33f-48e3ee052186-host-cni-netd\") pod \"ovnkube-node-sr2wt\" (UID: \"45027dd2-2a4d-4ff5-b33f-48e3ee052186\") " pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.755789 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.755779 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45027dd2-2a4d-4ff5-b33f-48e3ee052186-run-openvswitch\") pod \"ovnkube-node-sr2wt\" (UID: \"45027dd2-2a4d-4ff5-b33f-48e3ee052186\") " pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.755924 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.755790 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/45027dd2-2a4d-4ff5-b33f-48e3ee052186-systemd-units\") pod \"ovnkube-node-sr2wt\" (UID: \"45027dd2-2a4d-4ff5-b33f-48e3ee052186\") " pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.755924 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.755811 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/45027dd2-2a4d-4ff5-b33f-48e3ee052186-node-log\") pod \"ovnkube-node-sr2wt\" (UID: \"45027dd2-2a4d-4ff5-b33f-48e3ee052186\") " pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.755924 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.755831 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/45027dd2-2a4d-4ff5-b33f-48e3ee052186-host-cni-netd\") pod \"ovnkube-node-sr2wt\" (UID: \"45027dd2-2a4d-4ff5-b33f-48e3ee052186\") " pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.755924 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.755831 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45027dd2-2a4d-4ff5-b33f-48e3ee052186-host-run-ovn-kubernetes\") pod \"ovnkube-node-sr2wt\" (UID: \"45027dd2-2a4d-4ff5-b33f-48e3ee052186\") " pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.755924 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.755859 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/45027dd2-2a4d-4ff5-b33f-48e3ee052186-ovnkube-script-lib\") pod \"ovnkube-node-sr2wt\" (UID: \"45027dd2-2a4d-4ff5-b33f-48e3ee052186\") " pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.755924 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.755867 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/45027dd2-2a4d-4ff5-b33f-48e3ee052186-node-log\") pod \"ovnkube-node-sr2wt\" (UID: \"45027dd2-2a4d-4ff5-b33f-48e3ee052186\") " pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.755924 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.755879 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45027dd2-2a4d-4ff5-b33f-48e3ee052186-run-openvswitch\") pod \"ovnkube-node-sr2wt\" (UID: \"45027dd2-2a4d-4ff5-b33f-48e3ee052186\") " pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.755924 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.755898 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45027dd2-2a4d-4ff5-b33f-48e3ee052186-host-run-ovn-kubernetes\") pod \"ovnkube-node-sr2wt\" (UID: \"45027dd2-2a4d-4ff5-b33f-48e3ee052186\") " pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.755924 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.755888 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/45027dd2-2a4d-4ff5-b33f-48e3ee052186-host-kubelet\") pod \"ovnkube-node-sr2wt\" (UID: \"45027dd2-2a4d-4ff5-b33f-48e3ee052186\") " pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.755924 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.755927 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/45027dd2-2a4d-4ff5-b33f-48e3ee052186-host-kubelet\") pod \"ovnkube-node-sr2wt\" (UID: \"45027dd2-2a4d-4ff5-b33f-48e3ee052186\") " pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.756295 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.755958 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71722934-df74-4aa7-91e7-9f388d432991-tmp-dir\") pod \"node-resolver-lkfnm\" (UID: \"71722934-df74-4aa7-91e7-9f388d432991\") " pod="openshift-dns/node-resolver-lkfnm" May 21 11:30:46.756295 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.755988 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-r7n49\" (UniqueName: \"kubernetes.io/projected/45027dd2-2a4d-4ff5-b33f-48e3ee052186-kube-api-access-r7n49\") pod \"ovnkube-node-sr2wt\" (UID: \"45027dd2-2a4d-4ff5-b33f-48e3ee052186\") " pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.756295 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.756007 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45027dd2-2a4d-4ff5-b33f-48e3ee052186-var-lib-openvswitch\") pod \"ovnkube-node-sr2wt\" (UID: \"45027dd2-2a4d-4ff5-b33f-48e3ee052186\") " pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.756295 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.756023 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/414e12fe-b9ac-4bbf-8470-ad442215f5cb-metrics-certs\") pod \"network-metrics-daemon-xf5gc\" (UID: \"414e12fe-b9ac-4bbf-8470-ad442215f5cb\") " pod="openshift-multus/network-metrics-daemon-xf5gc" May 21 11:30:46.756295 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.756040 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kvd92\" (UniqueName: \"kubernetes.io/projected/8316b46b-bb73-4b4e-805b-da0101c1b222-kube-api-access-kvd92\") pod \"network-check-target-vw6wd\" (UID: \"8316b46b-bb73-4b4e-805b-da0101c1b222\") " pod="openshift-network-diagnostics/network-check-target-vw6wd" May 21 11:30:46.756295 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.756063 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/449c0696-add7-49fb-9892-5c100cdc6d7a-konnectivity-ca\") pod \"konnectivity-agent-w8vhb\" (UID: \"449c0696-add7-49fb-9892-5c100cdc6d7a\") " pod="kube-system/konnectivity-agent-w8vhb" May 21 11:30:46.756295 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.756095 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45027dd2-2a4d-4ff5-b33f-48e3ee052186-var-lib-openvswitch\") pod \"ovnkube-node-sr2wt\" (UID: \"45027dd2-2a4d-4ff5-b33f-48e3ee052186\") " pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.756295 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.756234 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45027dd2-2a4d-4ff5-b33f-48e3ee052186-etc-openvswitch\") pod \"ovnkube-node-sr2wt\" (UID: \"45027dd2-2a4d-4ff5-b33f-48e3ee052186\") " pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.756295 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.756263 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/449c0696-add7-49fb-9892-5c100cdc6d7a-agent-certs\") pod \"konnectivity-agent-w8vhb\" (UID: \"449c0696-add7-49fb-9892-5c100cdc6d7a\") " pod="kube-system/konnectivity-agent-w8vhb" May 21 11:30:46.756295 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.756259 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71722934-df74-4aa7-91e7-9f388d432991-tmp-dir\") pod \"node-resolver-lkfnm\" (UID: \"71722934-df74-4aa7-91e7-9f388d432991\") " pod="openshift-dns/node-resolver-lkfnm" May 21 11:30:46.756295 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.756280 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/45027dd2-2a4d-4ff5-b33f-48e3ee052186-host-run-netns\") pod \"ovnkube-node-sr2wt\" (UID: \"45027dd2-2a4d-4ff5-b33f-48e3ee052186\") " pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.756790 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.756304 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/45027dd2-2a4d-4ff5-b33f-48e3ee052186-host-run-netns\") pod \"ovnkube-node-sr2wt\" (UID: \"45027dd2-2a4d-4ff5-b33f-48e3ee052186\") " pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.756790 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.756312 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45027dd2-2a4d-4ff5-b33f-48e3ee052186-etc-openvswitch\") pod \"ovnkube-node-sr2wt\" (UID: \"45027dd2-2a4d-4ff5-b33f-48e3ee052186\") " pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.756790 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.756313 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/45027dd2-2a4d-4ff5-b33f-48e3ee052186-run-systemd\") pod \"ovnkube-node-sr2wt\" (UID: \"45027dd2-2a4d-4ff5-b33f-48e3ee052186\") " pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.756790 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.756348 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/45027dd2-2a4d-4ff5-b33f-48e3ee052186-run-systemd\") pod \"ovnkube-node-sr2wt\" (UID: \"45027dd2-2a4d-4ff5-b33f-48e3ee052186\") " pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.756790 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.756377 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/45027dd2-2a4d-4ff5-b33f-48e3ee052186-ovnkube-config\") pod \"ovnkube-node-sr2wt\" (UID: \"45027dd2-2a4d-4ff5-b33f-48e3ee052186\") " pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.756790 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.756415 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/45027dd2-2a4d-4ff5-b33f-48e3ee052186-host-slash\") pod \"ovnkube-node-sr2wt\" (UID: \"45027dd2-2a4d-4ff5-b33f-48e3ee052186\") " pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.756790 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:46.756432 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 21 11:30:46.756790 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.756442 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/45027dd2-2a4d-4ff5-b33f-48e3ee052186-env-overrides\") pod \"ovnkube-node-sr2wt\" (UID: \"45027dd2-2a4d-4ff5-b33f-48e3ee052186\") " pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.756790 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.756462 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/45027dd2-2a4d-4ff5-b33f-48e3ee052186-ovnkube-script-lib\") pod \"ovnkube-node-sr2wt\" (UID: \"45027dd2-2a4d-4ff5-b33f-48e3ee052186\") " pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.756790 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.756471 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45027dd2-2a4d-4ff5-b33f-48e3ee052186-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sr2wt\" (UID: \"45027dd2-2a4d-4ff5-b33f-48e3ee052186\") " pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.756790 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:46.756500 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/414e12fe-b9ac-4bbf-8470-ad442215f5cb-metrics-certs podName:414e12fe-b9ac-4bbf-8470-ad442215f5cb nodeName:}" failed. No retries permitted until 2026-05-21 11:30:47.256469052 +0000 UTC m=+2.151752605 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/414e12fe-b9ac-4bbf-8470-ad442215f5cb-metrics-certs") pod "network-metrics-daemon-xf5gc" (UID: "414e12fe-b9ac-4bbf-8470-ad442215f5cb") : object "openshift-multus"/"metrics-daemon-secret" not registered May 21 11:30:46.756790 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.756516 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45027dd2-2a4d-4ff5-b33f-48e3ee052186-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sr2wt\" (UID: \"45027dd2-2a4d-4ff5-b33f-48e3ee052186\") " pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.756790 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.756524 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/45027dd2-2a4d-4ff5-b33f-48e3ee052186-host-slash\") pod \"ovnkube-node-sr2wt\" (UID: \"45027dd2-2a4d-4ff5-b33f-48e3ee052186\") " pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.756790 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.756536 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/45027dd2-2a4d-4ff5-b33f-48e3ee052186-ovn-node-metrics-cert\") pod \"ovnkube-node-sr2wt\" (UID: \"45027dd2-2a4d-4ff5-b33f-48e3ee052186\") " pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.756790 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.756564 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/45027dd2-2a4d-4ff5-b33f-48e3ee052186-run-ovn\") pod \"ovnkube-node-sr2wt\" (UID: \"45027dd2-2a4d-4ff5-b33f-48e3ee052186\") " pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.756790 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.756585 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/45027dd2-2a4d-4ff5-b33f-48e3ee052186-host-cni-bin\") pod \"ovnkube-node-sr2wt\" (UID: \"45027dd2-2a4d-4ff5-b33f-48e3ee052186\") " pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.756790 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.756602 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hdsfn\" (UniqueName: \"kubernetes.io/projected/71722934-df74-4aa7-91e7-9f388d432991-kube-api-access-hdsfn\") pod \"node-resolver-lkfnm\" (UID: \"71722934-df74-4aa7-91e7-9f388d432991\") " pod="openshift-dns/node-resolver-lkfnm" May 21 11:30:46.757613 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.756630 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/45027dd2-2a4d-4ff5-b33f-48e3ee052186-log-socket\") pod \"ovnkube-node-sr2wt\" (UID: \"45027dd2-2a4d-4ff5-b33f-48e3ee052186\") " pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.757613 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.756656 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/71722934-df74-4aa7-91e7-9f388d432991-hosts-file\") pod \"node-resolver-lkfnm\" (UID: \"71722934-df74-4aa7-91e7-9f388d432991\") " pod="openshift-dns/node-resolver-lkfnm" May 21 11:30:46.757613 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.756666 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/45027dd2-2a4d-4ff5-b33f-48e3ee052186-host-cni-bin\") pod \"ovnkube-node-sr2wt\" (UID: \"45027dd2-2a4d-4ff5-b33f-48e3ee052186\") " pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.757613 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.756720 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/45027dd2-2a4d-4ff5-b33f-48e3ee052186-log-socket\") pod \"ovnkube-node-sr2wt\" (UID: \"45027dd2-2a4d-4ff5-b33f-48e3ee052186\") " pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.757613 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.756632 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/45027dd2-2a4d-4ff5-b33f-48e3ee052186-run-ovn\") pod \"ovnkube-node-sr2wt\" (UID: \"45027dd2-2a4d-4ff5-b33f-48e3ee052186\") " pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.757613 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.756850 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/71722934-df74-4aa7-91e7-9f388d432991-hosts-file\") pod \"node-resolver-lkfnm\" (UID: \"71722934-df74-4aa7-91e7-9f388d432991\") " pod="openshift-dns/node-resolver-lkfnm" May 21 11:30:46.757613 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.756942 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/45027dd2-2a4d-4ff5-b33f-48e3ee052186-env-overrides\") pod \"ovnkube-node-sr2wt\" (UID: \"45027dd2-2a4d-4ff5-b33f-48e3ee052186\") " pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.757613 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.757030 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/45027dd2-2a4d-4ff5-b33f-48e3ee052186-ovnkube-config\") pod \"ovnkube-node-sr2wt\" (UID: \"45027dd2-2a4d-4ff5-b33f-48e3ee052186\") " pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.757613 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.757323 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/449c0696-add7-49fb-9892-5c100cdc6d7a-konnectivity-ca\") pod \"konnectivity-agent-w8vhb\" (UID: \"449c0696-add7-49fb-9892-5c100cdc6d7a\") " pod="kube-system/konnectivity-agent-w8vhb" May 21 11:30:46.758804 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.758781 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/449c0696-add7-49fb-9892-5c100cdc6d7a-agent-certs\") pod \"konnectivity-agent-w8vhb\" (UID: \"449c0696-add7-49fb-9892-5c100cdc6d7a\") " pod="kube-system/konnectivity-agent-w8vhb" May 21 11:30:46.759184 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.759166 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/45027dd2-2a4d-4ff5-b33f-48e3ee052186-ovn-node-metrics-cert\") pod \"ovnkube-node-sr2wt\" (UID: \"45027dd2-2a4d-4ff5-b33f-48e3ee052186\") " pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.767379 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:46.767358 2564 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 21 11:30:46.767496 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:46.767383 2564 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 21 11:30:46.767496 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:46.767399 2564 projected.go:194] Error preparing data for projected volume kube-api-access-kvd92 for pod openshift-network-diagnostics/network-check-target-vw6wd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 21 11:30:46.767496 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:46.767472 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8316b46b-bb73-4b4e-805b-da0101c1b222-kube-api-access-kvd92 podName:8316b46b-bb73-4b4e-805b-da0101c1b222 nodeName:}" failed. No retries permitted until 2026-05-21 11:30:47.267450451 +0000 UTC m=+2.162733994 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-kvd92" (UniqueName: "kubernetes.io/projected/8316b46b-bb73-4b4e-805b-da0101c1b222-kube-api-access-kvd92") pod "network-check-target-vw6wd" (UID: "8316b46b-bb73-4b4e-805b-da0101c1b222") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 21 11:30:46.769069 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.769047 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqt78\" (UniqueName: \"kubernetes.io/projected/414e12fe-b9ac-4bbf-8470-ad442215f5cb-kube-api-access-cqt78\") pod \"network-metrics-daemon-xf5gc\" (UID: \"414e12fe-b9ac-4bbf-8470-ad442215f5cb\") " pod="openshift-multus/network-metrics-daemon-xf5gc" May 21 11:30:46.769449 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.769433 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdsfn\" (UniqueName: \"kubernetes.io/projected/71722934-df74-4aa7-91e7-9f388d432991-kube-api-access-hdsfn\") pod \"node-resolver-lkfnm\" (UID: \"71722934-df74-4aa7-91e7-9f388d432991\") " pod="openshift-dns/node-resolver-lkfnm" May 21 11:30:46.769888 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.769862 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7n49\" (UniqueName: \"kubernetes.io/projected/45027dd2-2a4d-4ff5-b33f-48e3ee052186-kube-api-access-r7n49\") pod \"ovnkube-node-sr2wt\" (UID: \"45027dd2-2a4d-4ff5-b33f-48e3ee052186\") " pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.862794 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.862766 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2l94" May 21 11:30:46.869464 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:46.869435 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5516274d_6162_4809_bfc0_1f5d4b850794.slice/crio-73220a0d90724db540b3b1411c3e4f34767fd61369c63ac931f727563f4b17b9 WatchSource:0}: Error finding container 73220a0d90724db540b3b1411c3e4f34767fd61369c63ac931f727563f4b17b9: Status 404 returned error can't find the container with id 73220a0d90724db540b3b1411c3e4f34767fd61369c63ac931f727563f4b17b9 May 21 11:30:46.876404 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.876382 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-lbcnh" May 21 11:30:46.881922 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:46.881899 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0c35189_f247_439b_9a5e_d199d3963920.slice/crio-5eeda1abf0897062dc5e299b815382b66c0b820c025f66f00c185042bb6fd2f7 WatchSource:0}: Error finding container 5eeda1abf0897062dc5e299b815382b66c0b820c025f66f00c185042bb6fd2f7: Status 404 returned error can't find the container with id 5eeda1abf0897062dc5e299b815382b66c0b820c025f66f00c185042bb6fd2f7 May 21 11:30:46.893538 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.893518 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-29k74" May 21 11:30:46.898064 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.898041 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9nh44" May 21 11:30:46.899500 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:46.899472 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fd57831_cff3_4f96_8695_cb4d02bc95a9.slice/crio-82c1ba0be21e5721ffd68856b5f252b57781b688464e15b1bc85ddef74905406 WatchSource:0}: Error finding container 82c1ba0be21e5721ffd68856b5f252b57781b688464e15b1bc85ddef74905406: Status 404 returned error can't find the container with id 82c1ba0be21e5721ffd68856b5f252b57781b688464e15b1bc85ddef74905406 May 21 11:30:46.905047 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:46.905027 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfa04605_9b0c_4c9b_a808_598623c60d2d.slice/crio-707a0b31dafad56ccac23758a44726e3cb5bcd707a5ee180349433d65f0d1d38 WatchSource:0}: Error finding container 707a0b31dafad56ccac23758a44726e3cb5bcd707a5ee180349433d65f0d1d38: Status 404 returned error can't find the container with id 707a0b31dafad56ccac23758a44726e3cb5bcd707a5ee180349433d65f0d1d38 May 21 11:30:46.918921 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.918902 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-q4k9w" May 21 11:30:46.925943 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:46.925918 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfceead7c_3987_4250_816f_e89182d3b315.slice/crio-d3f60dabff9133e278b9dbdab526e3925c771742bbe41d05bf1afce32c7801c4 WatchSource:0}: Error finding container d3f60dabff9133e278b9dbdab526e3925c771742bbe41d05bf1afce32c7801c4: Status 404 returned error can't find the container with id d3f60dabff9133e278b9dbdab526e3925c771742bbe41d05bf1afce32c7801c4 May 21 11:30:46.937145 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.937130 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-btjpv" May 21 11:30:46.942816 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.942800 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:30:46.943280 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:46.943249 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3fb34bd_0e60_47f6_9fa6_2bd541ce9deb.slice/crio-135e41fa31e2add6cd7330f60b2af6fae438c5c96c313ee2e5a9a2b011cb27e0 WatchSource:0}: Error finding container 135e41fa31e2add6cd7330f60b2af6fae438c5c96c313ee2e5a9a2b011cb27e0: Status 404 returned error can't find the container with id 135e41fa31e2add6cd7330f60b2af6fae438c5c96c313ee2e5a9a2b011cb27e0 May 21 11:30:46.948589 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:46.948568 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45027dd2_2a4d_4ff5_b33f_48e3ee052186.slice/crio-1abcfa9eac48847c229033cdaee86a844ed04d31e666d573e936b7c309a0f161 WatchSource:0}: Error finding container 1abcfa9eac48847c229033cdaee86a844ed04d31e666d573e936b7c309a0f161: Status 404 returned error can't find the container with id 1abcfa9eac48847c229033cdaee86a844ed04d31e666d573e936b7c309a0f161 May 21 11:30:46.967988 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.967965 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lkfnm" May 21 11:30:46.969462 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.969450 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-w8vhb" May 21 11:30:46.975089 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:46.975049 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71722934_df74_4aa7_91e7_9f388d432991.slice/crio-67f5e736ab67fef4d7b6ccb2c4687771c13390e969a182e120a7f472a4abdad2 WatchSource:0}: Error finding container 67f5e736ab67fef4d7b6ccb2c4687771c13390e969a182e120a7f472a4abdad2: Status 404 returned error can't find the container with id 67f5e736ab67fef4d7b6ccb2c4687771c13390e969a182e120a7f472a4abdad2 May 21 11:30:46.975971 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:30:46.975949 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod449c0696_add7_49fb_9892_5c100cdc6d7a.slice/crio-489af205a001c837e42893c0a237f852231174ddc475eed3513d8a0a9dafad99 WatchSource:0}: Error finding container 489af205a001c837e42893c0a237f852231174ddc475eed3513d8a0a9dafad99: Status 404 returned error can't find the container with id 489af205a001c837e42893c0a237f852231174ddc475eed3513d8a0a9dafad99 May 21 11:30:46.997803 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:46.997775 2564 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" May 21 11:30:47.259580 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:47.259495 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/414e12fe-b9ac-4bbf-8470-ad442215f5cb-metrics-certs\") pod \"network-metrics-daemon-xf5gc\" (UID: \"414e12fe-b9ac-4bbf-8470-ad442215f5cb\") " pod="openshift-multus/network-metrics-daemon-xf5gc" May 21 11:30:47.259739 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:47.259709 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 21 11:30:47.259800 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:47.259773 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/414e12fe-b9ac-4bbf-8470-ad442215f5cb-metrics-certs podName:414e12fe-b9ac-4bbf-8470-ad442215f5cb nodeName:}" failed. No retries permitted until 2026-05-21 11:30:48.259752664 +0000 UTC m=+3.155036204 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/414e12fe-b9ac-4bbf-8470-ad442215f5cb-metrics-certs") pod "network-metrics-daemon-xf5gc" (UID: "414e12fe-b9ac-4bbf-8470-ad442215f5cb") : object "openshift-multus"/"metrics-daemon-secret" not registered May 21 11:30:47.360096 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:47.360050 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kvd92\" (UniqueName: \"kubernetes.io/projected/8316b46b-bb73-4b4e-805b-da0101c1b222-kube-api-access-kvd92\") pod \"network-check-target-vw6wd\" (UID: \"8316b46b-bb73-4b4e-805b-da0101c1b222\") " pod="openshift-network-diagnostics/network-check-target-vw6wd" May 21 11:30:47.360290 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:47.360270 2564 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 21 11:30:47.360357 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:47.360300 2564 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 21 11:30:47.360357 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:47.360313 2564 projected.go:194] Error preparing data for projected volume kube-api-access-kvd92 for pod openshift-network-diagnostics/network-check-target-vw6wd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 21 11:30:47.360458 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:47.360375 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8316b46b-bb73-4b4e-805b-da0101c1b222-kube-api-access-kvd92 podName:8316b46b-bb73-4b4e-805b-da0101c1b222 nodeName:}" failed. No retries permitted until 2026-05-21 11:30:48.360355929 +0000 UTC m=+3.255639465 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-kvd92" (UniqueName: "kubernetes.io/projected/8316b46b-bb73-4b4e-805b-da0101c1b222-kube-api-access-kvd92") pod "network-check-target-vw6wd" (UID: "8316b46b-bb73-4b4e-805b-da0101c1b222") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 21 11:30:47.596985 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:47.596939 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-20 11:25:46 +0000 UTC" deadline="2028-01-30 04:15:26.467081338 +0000 UTC" May 21 11:30:47.596985 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:47.596984 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14848h44m38.870102339s" May 21 11:30:47.701899 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:47.701820 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-213.ec2.internal" event={"ID":"3da4ba4cdfe6fc200fbc8989e0319648","Type":"ContainerStarted","Data":"a87c6ae30ffe1998927010c65f3373f717d1cd7fcdf63647dc7b1b84cd1c482c"} May 21 11:30:47.704288 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:47.704206 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-213.ec2.internal" event={"ID":"0694ef6a59bf20f80056d0d4f3f2967d","Type":"ContainerStarted","Data":"6bfb782524b2eb154b63735c8c97516b2bcc4a01d7ce9c0290f8fa91882b8ea7"} May 21 11:30:47.724426 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:47.724337 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" event={"ID":"45027dd2-2a4d-4ff5-b33f-48e3ee052186","Type":"ContainerStarted","Data":"1abcfa9eac48847c229033cdaee86a844ed04d31e666d573e936b7c309a0f161"} May 21 11:30:47.741273 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:47.741124 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-btjpv" event={"ID":"b3fb34bd-0e60-47f6-9fa6-2bd541ce9deb","Type":"ContainerStarted","Data":"135e41fa31e2add6cd7330f60b2af6fae438c5c96c313ee2e5a9a2b011cb27e0"} May 21 11:30:47.763766 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:47.763729 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-q4k9w" event={"ID":"fceead7c-3987-4250-816f-e89182d3b315","Type":"ContainerStarted","Data":"d3f60dabff9133e278b9dbdab526e3925c771742bbe41d05bf1afce32c7801c4"} May 21 11:30:47.774747 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:47.772470 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-29k74" event={"ID":"0fd57831-cff3-4f96-8695-cb4d02bc95a9","Type":"ContainerStarted","Data":"82c1ba0be21e5721ffd68856b5f252b57781b688464e15b1bc85ddef74905406"} May 21 11:30:47.774747 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:47.774623 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-lbcnh" event={"ID":"e0c35189-f247-439b-9a5e-d199d3963920","Type":"ContainerStarted","Data":"5eeda1abf0897062dc5e299b815382b66c0b820c025f66f00c185042bb6fd2f7"} May 21 11:30:47.783674 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:47.783594 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-w8vhb" event={"ID":"449c0696-add7-49fb-9892-5c100cdc6d7a","Type":"ContainerStarted","Data":"489af205a001c837e42893c0a237f852231174ddc475eed3513d8a0a9dafad99"} May 21 11:30:47.788334 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:47.788286 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lkfnm" event={"ID":"71722934-df74-4aa7-91e7-9f388d432991","Type":"ContainerStarted","Data":"67f5e736ab67fef4d7b6ccb2c4687771c13390e969a182e120a7f472a4abdad2"} May 21 11:30:47.801019 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:47.800992 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9nh44" event={"ID":"bfa04605-9b0c-4c9b-a808-598623c60d2d","Type":"ContainerStarted","Data":"707a0b31dafad56ccac23758a44726e3cb5bcd707a5ee180349433d65f0d1d38"} May 21 11:30:47.804440 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:47.804413 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2l94" event={"ID":"5516274d-6162-4809-bfc0-1f5d4b850794","Type":"ContainerStarted","Data":"73220a0d90724db540b3b1411c3e4f34767fd61369c63ac931f727563f4b17b9"} May 21 11:30:47.842552 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:47.842521 2564 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" May 21 11:30:47.993873 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:47.993773 2564 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" May 21 11:30:48.267582 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:48.267498 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/414e12fe-b9ac-4bbf-8470-ad442215f5cb-metrics-certs\") pod \"network-metrics-daemon-xf5gc\" (UID: \"414e12fe-b9ac-4bbf-8470-ad442215f5cb\") " pod="openshift-multus/network-metrics-daemon-xf5gc" May 21 11:30:48.267751 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:48.267706 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 21 11:30:48.267814 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:48.267773 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/414e12fe-b9ac-4bbf-8470-ad442215f5cb-metrics-certs podName:414e12fe-b9ac-4bbf-8470-ad442215f5cb nodeName:}" failed. No retries permitted until 2026-05-21 11:30:50.267752921 +0000 UTC m=+5.163036459 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/414e12fe-b9ac-4bbf-8470-ad442215f5cb-metrics-certs") pod "network-metrics-daemon-xf5gc" (UID: "414e12fe-b9ac-4bbf-8470-ad442215f5cb") : object "openshift-multus"/"metrics-daemon-secret" not registered May 21 11:30:48.368034 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:48.368002 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kvd92\" (UniqueName: \"kubernetes.io/projected/8316b46b-bb73-4b4e-805b-da0101c1b222-kube-api-access-kvd92\") pod \"network-check-target-vw6wd\" (UID: \"8316b46b-bb73-4b4e-805b-da0101c1b222\") " pod="openshift-network-diagnostics/network-check-target-vw6wd" May 21 11:30:48.368255 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:48.368213 2564 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 21 11:30:48.368255 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:48.368238 2564 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 21 11:30:48.368255 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:48.368251 2564 projected.go:194] Error preparing data for projected volume kube-api-access-kvd92 for pod openshift-network-diagnostics/network-check-target-vw6wd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 21 11:30:48.368377 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:48.368307 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8316b46b-bb73-4b4e-805b-da0101c1b222-kube-api-access-kvd92 podName:8316b46b-bb73-4b4e-805b-da0101c1b222 nodeName:}" failed. No retries permitted until 2026-05-21 11:30:50.368288278 +0000 UTC m=+5.263571816 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-kvd92" (UniqueName: "kubernetes.io/projected/8316b46b-bb73-4b4e-805b-da0101c1b222-kube-api-access-kvd92") pod "network-check-target-vw6wd" (UID: "8316b46b-bb73-4b4e-805b-da0101c1b222") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 21 11:30:48.597446 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:48.597398 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-20 11:25:46 +0000 UTC" deadline="2028-01-27 08:06:33.094911038 +0000 UTC" May 21 11:30:48.597446 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:48.597440 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14780h35m44.497475342s" May 21 11:30:48.679675 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:48.679640 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vw6wd" May 21 11:30:48.679853 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:48.679769 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vw6wd" podUID="8316b46b-bb73-4b4e-805b-da0101c1b222" May 21 11:30:48.680245 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:48.680220 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xf5gc" May 21 11:30:48.680357 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:48.680334 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xf5gc" podUID="414e12fe-b9ac-4bbf-8470-ad442215f5cb" May 21 11:30:48.811114 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:48.811070 2564 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" May 21 11:30:50.286148 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:50.286115 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/414e12fe-b9ac-4bbf-8470-ad442215f5cb-metrics-certs\") pod \"network-metrics-daemon-xf5gc\" (UID: \"414e12fe-b9ac-4bbf-8470-ad442215f5cb\") " pod="openshift-multus/network-metrics-daemon-xf5gc" May 21 11:30:50.286605 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:50.286328 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 21 11:30:50.286605 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:50.286393 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/414e12fe-b9ac-4bbf-8470-ad442215f5cb-metrics-certs podName:414e12fe-b9ac-4bbf-8470-ad442215f5cb nodeName:}" failed. No retries permitted until 2026-05-21 11:30:54.286373549 +0000 UTC m=+9.181657075 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/414e12fe-b9ac-4bbf-8470-ad442215f5cb-metrics-certs") pod "network-metrics-daemon-xf5gc" (UID: "414e12fe-b9ac-4bbf-8470-ad442215f5cb") : object "openshift-multus"/"metrics-daemon-secret" not registered May 21 11:30:50.387318 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:50.387275 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kvd92\" (UniqueName: \"kubernetes.io/projected/8316b46b-bb73-4b4e-805b-da0101c1b222-kube-api-access-kvd92\") pod \"network-check-target-vw6wd\" (UID: \"8316b46b-bb73-4b4e-805b-da0101c1b222\") " pod="openshift-network-diagnostics/network-check-target-vw6wd" May 21 11:30:50.387463 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:50.387446 2564 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 21 11:30:50.387513 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:50.387469 2564 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 21 11:30:50.387513 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:50.387483 2564 projected.go:194] Error preparing data for projected volume kube-api-access-kvd92 for pod openshift-network-diagnostics/network-check-target-vw6wd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 21 11:30:50.387587 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:50.387538 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8316b46b-bb73-4b4e-805b-da0101c1b222-kube-api-access-kvd92 podName:8316b46b-bb73-4b4e-805b-da0101c1b222 nodeName:}" failed. No retries permitted until 2026-05-21 11:30:54.38752065 +0000 UTC m=+9.282804188 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-kvd92" (UniqueName: "kubernetes.io/projected/8316b46b-bb73-4b4e-805b-da0101c1b222-kube-api-access-kvd92") pod "network-check-target-vw6wd" (UID: "8316b46b-bb73-4b4e-805b-da0101c1b222") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 21 11:30:50.679800 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:50.679764 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vw6wd" May 21 11:30:50.679969 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:50.679878 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vw6wd" podUID="8316b46b-bb73-4b4e-805b-da0101c1b222" May 21 11:30:50.679969 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:50.679959 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xf5gc" May 21 11:30:50.680093 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:50.680037 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xf5gc" podUID="414e12fe-b9ac-4bbf-8470-ad442215f5cb" May 21 11:30:52.679664 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:52.679627 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xf5gc" May 21 11:30:52.680164 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:52.679781 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xf5gc" podUID="414e12fe-b9ac-4bbf-8470-ad442215f5cb" May 21 11:30:52.680229 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:52.680217 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vw6wd" May 21 11:30:52.680334 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:52.680313 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vw6wd" podUID="8316b46b-bb73-4b4e-805b-da0101c1b222" May 21 11:30:54.321478 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:54.320841 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/414e12fe-b9ac-4bbf-8470-ad442215f5cb-metrics-certs\") pod \"network-metrics-daemon-xf5gc\" (UID: \"414e12fe-b9ac-4bbf-8470-ad442215f5cb\") " pod="openshift-multus/network-metrics-daemon-xf5gc" May 21 11:30:54.321478 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:54.321011 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 21 11:30:54.321478 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:54.321096 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/414e12fe-b9ac-4bbf-8470-ad442215f5cb-metrics-certs podName:414e12fe-b9ac-4bbf-8470-ad442215f5cb nodeName:}" failed. No retries permitted until 2026-05-21 11:31:02.321058725 +0000 UTC m=+17.216342257 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/414e12fe-b9ac-4bbf-8470-ad442215f5cb-metrics-certs") pod "network-metrics-daemon-xf5gc" (UID: "414e12fe-b9ac-4bbf-8470-ad442215f5cb") : object "openshift-multus"/"metrics-daemon-secret" not registered May 21 11:30:54.421580 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:54.421521 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kvd92\" (UniqueName: \"kubernetes.io/projected/8316b46b-bb73-4b4e-805b-da0101c1b222-kube-api-access-kvd92\") pod \"network-check-target-vw6wd\" (UID: \"8316b46b-bb73-4b4e-805b-da0101c1b222\") " pod="openshift-network-diagnostics/network-check-target-vw6wd" May 21 11:30:54.421748 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:54.421709 2564 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 21 11:30:54.421748 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:54.421735 2564 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 21 11:30:54.421748 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:54.421749 2564 projected.go:194] Error preparing data for projected volume kube-api-access-kvd92 for pod openshift-network-diagnostics/network-check-target-vw6wd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 21 11:30:54.421870 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:54.421810 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8316b46b-bb73-4b4e-805b-da0101c1b222-kube-api-access-kvd92 podName:8316b46b-bb73-4b4e-805b-da0101c1b222 nodeName:}" failed. No retries permitted until 2026-05-21 11:31:02.421791649 +0000 UTC m=+17.317075186 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-kvd92" (UniqueName: "kubernetes.io/projected/8316b46b-bb73-4b4e-805b-da0101c1b222-kube-api-access-kvd92") pod "network-check-target-vw6wd" (UID: "8316b46b-bb73-4b4e-805b-da0101c1b222") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 21 11:30:54.680025 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:54.679837 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vw6wd" May 21 11:30:54.680292 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:54.680135 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vw6wd" podUID="8316b46b-bb73-4b4e-805b-da0101c1b222" May 21 11:30:54.680620 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:54.680459 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xf5gc" May 21 11:30:54.680620 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:54.680578 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xf5gc" podUID="414e12fe-b9ac-4bbf-8470-ad442215f5cb" May 21 11:30:56.680158 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:56.679610 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xf5gc" May 21 11:30:56.680158 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:56.679608 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vw6wd" May 21 11:30:56.680158 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:56.679767 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xf5gc" podUID="414e12fe-b9ac-4bbf-8470-ad442215f5cb" May 21 11:30:56.680158 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:56.679808 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vw6wd" podUID="8316b46b-bb73-4b4e-805b-da0101c1b222" May 21 11:30:58.680244 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:58.680200 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xf5gc" May 21 11:30:58.680690 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:30:58.680200 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vw6wd" May 21 11:30:58.680690 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:58.680342 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xf5gc" podUID="414e12fe-b9ac-4bbf-8470-ad442215f5cb" May 21 11:30:58.680690 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:30:58.680377 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vw6wd" podUID="8316b46b-bb73-4b4e-805b-da0101c1b222" May 21 11:31:00.679768 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:00.679540 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vw6wd" May 21 11:31:00.680244 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:00.679584 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xf5gc" May 21 11:31:00.680244 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:00.679875 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vw6wd" podUID="8316b46b-bb73-4b4e-805b-da0101c1b222" May 21 11:31:00.680244 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:00.679980 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xf5gc" podUID="414e12fe-b9ac-4bbf-8470-ad442215f5cb" May 21 11:31:02.378163 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:02.378112 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/414e12fe-b9ac-4bbf-8470-ad442215f5cb-metrics-certs\") pod \"network-metrics-daemon-xf5gc\" (UID: \"414e12fe-b9ac-4bbf-8470-ad442215f5cb\") " pod="openshift-multus/network-metrics-daemon-xf5gc" May 21 11:31:02.378645 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:02.378270 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 21 11:31:02.378645 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:02.378353 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/414e12fe-b9ac-4bbf-8470-ad442215f5cb-metrics-certs podName:414e12fe-b9ac-4bbf-8470-ad442215f5cb nodeName:}" failed. No retries permitted until 2026-05-21 11:31:18.378332704 +0000 UTC m=+33.273616229 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/414e12fe-b9ac-4bbf-8470-ad442215f5cb-metrics-certs") pod "network-metrics-daemon-xf5gc" (UID: "414e12fe-b9ac-4bbf-8470-ad442215f5cb") : object "openshift-multus"/"metrics-daemon-secret" not registered May 21 11:31:02.479248 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:02.479213 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kvd92\" (UniqueName: \"kubernetes.io/projected/8316b46b-bb73-4b4e-805b-da0101c1b222-kube-api-access-kvd92\") pod \"network-check-target-vw6wd\" (UID: \"8316b46b-bb73-4b4e-805b-da0101c1b222\") " pod="openshift-network-diagnostics/network-check-target-vw6wd" May 21 11:31:02.479446 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:02.479384 2564 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 21 11:31:02.479446 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:02.479405 2564 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 21 11:31:02.479446 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:02.479417 2564 projected.go:194] Error preparing data for projected volume kube-api-access-kvd92 for pod openshift-network-diagnostics/network-check-target-vw6wd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 21 11:31:02.479606 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:02.479485 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8316b46b-bb73-4b4e-805b-da0101c1b222-kube-api-access-kvd92 podName:8316b46b-bb73-4b4e-805b-da0101c1b222 nodeName:}" failed. No retries permitted until 2026-05-21 11:31:18.479466803 +0000 UTC m=+33.374750343 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-kvd92" (UniqueName: "kubernetes.io/projected/8316b46b-bb73-4b4e-805b-da0101c1b222-kube-api-access-kvd92") pod "network-check-target-vw6wd" (UID: "8316b46b-bb73-4b4e-805b-da0101c1b222") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 21 11:31:02.680613 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:02.680044 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vw6wd" May 21 11:31:02.680613 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:02.680055 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xf5gc" May 21 11:31:02.680613 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:02.680209 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vw6wd" podUID="8316b46b-bb73-4b4e-805b-da0101c1b222" May 21 11:31:02.680613 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:02.680284 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xf5gc" podUID="414e12fe-b9ac-4bbf-8470-ad442215f5cb" May 21 11:31:04.679358 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:04.679317 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vw6wd" May 21 11:31:04.679813 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:04.679329 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xf5gc" May 21 11:31:04.679813 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:04.679458 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vw6wd" podUID="8316b46b-bb73-4b4e-805b-da0101c1b222" May 21 11:31:04.679813 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:04.679543 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xf5gc" podUID="414e12fe-b9ac-4bbf-8470-ad442215f5cb" May 21 11:31:06.679707 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:06.679462 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vw6wd" May 21 11:31:06.680463 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:06.679515 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xf5gc" May 21 11:31:06.680463 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:06.679822 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vw6wd" podUID="8316b46b-bb73-4b4e-805b-da0101c1b222" May 21 11:31:06.680463 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:06.679862 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xf5gc" podUID="414e12fe-b9ac-4bbf-8470-ad442215f5cb" May 21 11:31:06.846242 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:06.846200 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-213.ec2.internal" event={"ID":"3da4ba4cdfe6fc200fbc8989e0319648","Type":"ContainerStarted","Data":"87a736475c6c6c4634578fb0b1b18b52ea728ebd358609fbd4cb5086e67ec244"} May 21 11:31:06.846401 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:06.846389 2564 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-213.ec2.internal" May 21 11:31:06.848651 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:06.848631 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sr2wt_45027dd2-2a4d-4ff5-b33f-48e3ee052186/ovn-acl-logging/0.log" May 21 11:31:06.849286 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:06.849263 2564 generic.go:358] "Generic (PLEG): container finished" podID="45027dd2-2a4d-4ff5-b33f-48e3ee052186" containerID="3399cb022cf64f03d912805801d3c34e6a34c0ecd0e241577e40ba28ed18dc2d" exitCode=1 May 21 11:31:06.849383 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:06.849287 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" event={"ID":"45027dd2-2a4d-4ff5-b33f-48e3ee052186","Type":"ContainerStarted","Data":"7cbc48716932e4967f52e2cb196ffe66c3362e66fbfad0c7f7239c4f2ea888c5"} May 21 11:31:06.849383 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:06.849315 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" event={"ID":"45027dd2-2a4d-4ff5-b33f-48e3ee052186","Type":"ContainerStarted","Data":"fe3af5c7f13958bf01feb7afb538665c049a6f256dab1262209c7dff3e0e6d81"} May 21 11:31:06.849383 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:06.849327 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" event={"ID":"45027dd2-2a4d-4ff5-b33f-48e3ee052186","Type":"ContainerStarted","Data":"406aecb17e7f5620871d76b506209d17d11865d4ba1a41774e2c35076f51ccc3"} May 21 11:31:06.849383 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:06.849340 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" event={"ID":"45027dd2-2a4d-4ff5-b33f-48e3ee052186","Type":"ContainerStarted","Data":"3da6f0bd415642d650da2c52e71aac6942a9a5379216cf3681c6f2a81d7edceb"} May 21 11:31:06.849383 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:06.849348 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" event={"ID":"45027dd2-2a4d-4ff5-b33f-48e3ee052186","Type":"ContainerDied","Data":"3399cb022cf64f03d912805801d3c34e6a34c0ecd0e241577e40ba28ed18dc2d"} May 21 11:31:06.849383 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:06.849359 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" event={"ID":"45027dd2-2a4d-4ff5-b33f-48e3ee052186","Type":"ContainerStarted","Data":"6c4e7470ee7109d95c9abe6179d2f9a3bd23ebff6245c243351f4608cf20508e"} May 21 11:31:06.850626 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:06.850602 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-q4k9w" event={"ID":"fceead7c-3987-4250-816f-e89182d3b315","Type":"ContainerStarted","Data":"26fa02a6ee6ba0b1ef6222fbc32c4d702880eaafdc820883648ee181b3b60d66"} May 21 11:31:06.851778 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:06.851760 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-lbcnh" event={"ID":"e0c35189-f247-439b-9a5e-d199d3963920","Type":"ContainerStarted","Data":"7609275ed6dc5cc450b273755daac73705c2cc31ddde159a22a733f755ae5a3f"} May 21 11:31:06.858634 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:06.858607 2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" May 21 11:31:06.859393 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:06.859374 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-139-213.ec2.internal"] May 21 11:31:06.868598 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:06.868556 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-q4k9w" podStartSLOduration=3.035261624 podStartE2EDuration="21.868544524s" podCreationTimestamp="2026-05-21 11:30:45 +0000 UTC" firstStartedPulling="2026-05-21 11:30:46.927325351 +0000 UTC m=+1.822608874" lastFinishedPulling="2026-05-21 11:31:05.760608248 +0000 UTC m=+20.655891774" observedRunningTime="2026-05-21 11:31:06.868442575 +0000 UTC m=+21.763726120" watchObservedRunningTime="2026-05-21 11:31:06.868544524 +0000 UTC m=+21.763828069" May 21 11:31:06.902496 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:06.902460 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-213.ec2.internal" podStartSLOduration=0.902449381 podStartE2EDuration="902.449381ms" podCreationTimestamp="2026-05-21 11:31:06 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-21 11:31:06.902184618 +0000 UTC m=+21.797468165" watchObservedRunningTime="2026-05-21 11:31:06.902449381 +0000 UTC m=+21.797732926" May 21 11:31:06.902620 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:06.902600 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-lbcnh" podStartSLOduration=3.068147664 podStartE2EDuration="21.902595636s" podCreationTimestamp="2026-05-21 11:30:45 +0000 UTC" firstStartedPulling="2026-05-21 11:30:46.884480605 +0000 UTC m=+1.779764128" lastFinishedPulling="2026-05-21 11:31:05.718928567 +0000 UTC m=+20.614212100" observedRunningTime="2026-05-21 11:31:06.888361155 +0000 UTC m=+21.783644701" watchObservedRunningTime="2026-05-21 11:31:06.902595636 +0000 UTC m=+21.797879237" May 21 11:31:07.855439 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:07.855169 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-w8vhb" event={"ID":"449c0696-add7-49fb-9892-5c100cdc6d7a","Type":"ContainerStarted","Data":"744450b66ad8ded6e5883cfe1d8d1bb17b79476d14d35624576e5ab91f7863aa"} May 21 11:31:07.856730 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:07.856698 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lkfnm" event={"ID":"71722934-df74-4aa7-91e7-9f388d432991","Type":"ContainerStarted","Data":"67ab4cfa4b0d0fbee0ac01321bc5a684feb1370d8e752e498b84eafda7e000e8"} May 21 11:31:07.858392 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:07.858363 2564 generic.go:358] "Generic (PLEG): container finished" podID="bfa04605-9b0c-4c9b-a808-598623c60d2d" containerID="726e16632c385328cd83c5e035072fc91bb7fdd1c0ec0ecfe76d1041b2fea020" exitCode=0 May 21 11:31:07.858502 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:07.858440 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9nh44" event={"ID":"bfa04605-9b0c-4c9b-a808-598623c60d2d","Type":"ContainerDied","Data":"726e16632c385328cd83c5e035072fc91bb7fdd1c0ec0ecfe76d1041b2fea020"} May 21 11:31:07.860423 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:07.860399 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2l94" event={"ID":"5516274d-6162-4809-bfc0-1f5d4b850794","Type":"ContainerStarted","Data":"6567fb5bbe01aaf35be710e074cb5ac33d3c94e83a77926a5ada89522fd91424"} May 21 11:31:07.861973 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:07.861948 2564 generic.go:358] "Generic (PLEG): container finished" podID="0694ef6a59bf20f80056d0d4f3f2967d" containerID="5663229b63149871b63dfb66daf27c08368b971cc148209b7a996f649e669d8d" exitCode=0 May 21 11:31:07.862063 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:07.862034 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-213.ec2.internal" event={"ID":"0694ef6a59bf20f80056d0d4f3f2967d","Type":"ContainerDied","Data":"5663229b63149871b63dfb66daf27c08368b971cc148209b7a996f649e669d8d"} May 21 11:31:07.863656 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:07.863613 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-btjpv" event={"ID":"b3fb34bd-0e60-47f6-9fa6-2bd541ce9deb","Type":"ContainerStarted","Data":"d5be9990860390624953a8665688551b707a4a419c5a52bcbcd073ca67f5d912"} May 21 11:31:07.865373 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:07.865351 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-29k74" event={"ID":"0fd57831-cff3-4f96-8695-cb4d02bc95a9","Type":"ContainerStarted","Data":"b2533c45fd0a4a8b1ca921b04aabc7aacc4aaab2ca7f2fbff7f4873256416b70"} May 21 11:31:07.865527 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:07.865511 2564 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-213.ec2.internal" May 21 11:31:07.872188 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:07.872146 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-w8vhb" podStartSLOduration=8.60914969 podStartE2EDuration="22.872134393s" podCreationTimestamp="2026-05-21 11:30:45 +0000 UTC" firstStartedPulling="2026-05-21 11:30:46.977434913 +0000 UTC m=+1.872718436" lastFinishedPulling="2026-05-21 11:31:01.240419601 +0000 UTC m=+16.135703139" observedRunningTime="2026-05-21 11:31:07.871490857 +0000 UTC m=+22.766774403" watchObservedRunningTime="2026-05-21 11:31:07.872134393 +0000 UTC m=+22.767417940" May 21 11:31:07.875817 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:07.875801 2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" May 21 11:31:07.875910 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:07.875837 2564 kubelet.go:3342] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-139-213.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-213.ec2.internal" May 21 11:31:07.887614 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:07.887579 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-29k74" podStartSLOduration=4.07290123 podStartE2EDuration="22.887568176s" podCreationTimestamp="2026-05-21 11:30:45 +0000 UTC" firstStartedPulling="2026-05-21 11:30:46.901704801 +0000 UTC m=+1.796988327" lastFinishedPulling="2026-05-21 11:31:05.716371737 +0000 UTC m=+20.611655273" observedRunningTime="2026-05-21 11:31:07.887454722 +0000 UTC m=+22.782738268" watchObservedRunningTime="2026-05-21 11:31:07.887568176 +0000 UTC m=+22.782851721" May 21 11:31:07.937519 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:07.937469 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-lkfnm" podStartSLOduration=4.197948114 podStartE2EDuration="22.937455082s" podCreationTimestamp="2026-05-21 11:30:45 +0000 UTC" firstStartedPulling="2026-05-21 11:30:46.976770234 +0000 UTC m=+1.872053757" lastFinishedPulling="2026-05-21 11:31:05.716277189 +0000 UTC m=+20.611560725" observedRunningTime="2026-05-21 11:31:07.937279316 +0000 UTC m=+22.832562862" watchObservedRunningTime="2026-05-21 11:31:07.937455082 +0000 UTC m=+22.832738626" May 21 11:31:08.679696 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:08.679671 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vw6wd" May 21 11:31:08.679837 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:08.679679 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xf5gc" May 21 11:31:08.679837 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:08.679788 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vw6wd" podUID="8316b46b-bb73-4b4e-805b-da0101c1b222" May 21 11:31:08.679959 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:08.679881 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xf5gc" podUID="414e12fe-b9ac-4bbf-8470-ad442215f5cb" May 21 11:31:08.790880 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:08.790856 2564 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" May 21 11:31:08.868704 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:08.868667 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2l94" event={"ID":"5516274d-6162-4809-bfc0-1f5d4b850794","Type":"ContainerStarted","Data":"838dbf36462ad744f57d079404339c6141ab588b5b04b9e31f3fbf208d7ce8ae"} May 21 11:31:08.870586 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:08.870555 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-213.ec2.internal" event={"ID":"0694ef6a59bf20f80056d0d4f3f2967d","Type":"ContainerStarted","Data":"1fd2ab2f3ebf1227f9fb6d53c0b1ec88e663ee55dc7d5da8ef32b25fc7498e01"} May 21 11:31:08.873859 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:08.873834 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sr2wt_45027dd2-2a4d-4ff5-b33f-48e3ee052186/ovn-acl-logging/0.log" May 21 11:31:08.874347 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:08.874322 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" event={"ID":"45027dd2-2a4d-4ff5-b33f-48e3ee052186","Type":"ContainerStarted","Data":"8bc9bddf6654a8ce06fb167fae9a98bd4ce4bdb09a74221ea56ce33e25708cbe"} May 21 11:31:08.889394 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:08.889307 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-btjpv" podStartSLOduration=9.594411636 podStartE2EDuration="23.889293363s" podCreationTimestamp="2026-05-21 11:30:45 +0000 UTC" firstStartedPulling="2026-05-21 11:30:46.94557858 +0000 UTC m=+1.840862106" lastFinishedPulling="2026-05-21 11:31:01.240460292 +0000 UTC m=+16.135743833" observedRunningTime="2026-05-21 11:31:07.970400256 +0000 UTC m=+22.865683800" watchObservedRunningTime="2026-05-21 11:31:08.889293363 +0000 UTC m=+23.784576908" May 21 11:31:09.568560 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:09.568522 2564 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-w8vhb" May 21 11:31:09.569274 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:09.569253 2564 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-w8vhb" May 21 11:31:09.587149 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:09.587093 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-213.ec2.internal" podStartSLOduration=23.587052798 podStartE2EDuration="23.587052798s" podCreationTimestamp="2026-05-21 11:30:46 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-21 11:31:08.888971195 +0000 UTC m=+23.784254749" watchObservedRunningTime="2026-05-21 11:31:09.587052798 +0000 UTC m=+24.482336346" May 21 11:31:09.615872 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:09.615765 2564 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-05-21T11:31:08.790877849Z","UUID":"dd3424c3-fe0d-4083-88c0-7ffada763b03","Handler":null,"Name":"","Endpoint":""} May 21 11:31:09.618979 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:09.618954 2564 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 May 21 11:31:09.619120 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:09.618986 2564 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock May 21 11:31:09.876778 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:09.876691 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-w8vhb" May 21 11:31:09.877374 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:09.877353 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-w8vhb" May 21 11:31:10.679456 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:10.679281 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xf5gc" May 21 11:31:10.679629 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:10.679281 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vw6wd" May 21 11:31:10.679629 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:10.679535 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xf5gc" podUID="414e12fe-b9ac-4bbf-8470-ad442215f5cb" May 21 11:31:10.679629 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:10.679600 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vw6wd" podUID="8316b46b-bb73-4b4e-805b-da0101c1b222" May 21 11:31:10.880561 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:10.880530 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2l94" event={"ID":"5516274d-6162-4809-bfc0-1f5d4b850794","Type":"ContainerStarted","Data":"87faa661ab2a796b9f7d29496b965103f59423a50cbb58e78c5a449d50ce77ef"} May 21 11:31:10.901896 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:10.901854 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m2l94" podStartSLOduration=2.695593208 podStartE2EDuration="25.901838473s" podCreationTimestamp="2026-05-21 11:30:45 +0000 UTC" firstStartedPulling="2026-05-21 11:30:46.870971029 +0000 UTC m=+1.766254552" lastFinishedPulling="2026-05-21 11:31:10.07721628 +0000 UTC m=+24.972499817" observedRunningTime="2026-05-21 11:31:10.901353556 +0000 UTC m=+25.796637103" watchObservedRunningTime="2026-05-21 11:31:10.901838473 +0000 UTC m=+25.797122015" May 21 11:31:11.886925 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:11.886734 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sr2wt_45027dd2-2a4d-4ff5-b33f-48e3ee052186/ovn-acl-logging/0.log" May 21 11:31:11.887501 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:11.887267 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" event={"ID":"45027dd2-2a4d-4ff5-b33f-48e3ee052186","Type":"ContainerStarted","Data":"80c7bead7d5905893432b31466dab3e789ed208122f288f04df7666ff70e80a5"} May 21 11:31:11.887759 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:11.887685 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:31:11.887759 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:11.887740 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:31:11.888020 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:11.887768 2564 scope.go:117] "RemoveContainer" containerID="3399cb022cf64f03d912805801d3c34e6a34c0ecd0e241577e40ba28ed18dc2d" May 21 11:31:11.905186 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:11.905158 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:31:12.679537 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:12.679515 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vw6wd" May 21 11:31:12.679625 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:12.679517 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xf5gc" May 21 11:31:12.679625 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:12.679611 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vw6wd" podUID="8316b46b-bb73-4b4e-805b-da0101c1b222" May 21 11:31:12.679716 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:12.679698 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xf5gc" podUID="414e12fe-b9ac-4bbf-8470-ad442215f5cb" May 21 11:31:12.862426 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:12.862356 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vw6wd"] May 21 11:31:12.866714 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:12.866688 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xf5gc"] May 21 11:31:12.891516 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:12.891485 2564 generic.go:358] "Generic (PLEG): container finished" podID="bfa04605-9b0c-4c9b-a808-598623c60d2d" containerID="c2d1b294b52581092583ff6d09957da4afd3656561582c9605b0795a4e23c03a" exitCode=0 May 21 11:31:12.892030 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:12.891568 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9nh44" event={"ID":"bfa04605-9b0c-4c9b-a808-598623c60d2d","Type":"ContainerDied","Data":"c2d1b294b52581092583ff6d09957da4afd3656561582c9605b0795a4e23c03a"} May 21 11:31:12.895548 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:12.895531 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sr2wt_45027dd2-2a4d-4ff5-b33f-48e3ee052186/ovn-acl-logging/0.log" May 21 11:31:12.895979 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:12.895952 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" event={"ID":"45027dd2-2a4d-4ff5-b33f-48e3ee052186","Type":"ContainerStarted","Data":"6c9c143399a3dab63c29333ec6f1c527695dd910e58eafafdde81c0f96b06bea"} May 21 11:31:12.896118 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:12.895962 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xf5gc" May 21 11:31:12.896118 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:12.896003 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vw6wd" May 21 11:31:12.896232 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:12.896134 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xf5gc" podUID="414e12fe-b9ac-4bbf-8470-ad442215f5cb" May 21 11:31:12.896284 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:12.896250 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vw6wd" podUID="8316b46b-bb73-4b4e-805b-da0101c1b222" May 21 11:31:12.896284 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:12.896271 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:31:12.914544 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:12.914515 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:31:12.951827 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:12.951777 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" podStartSLOduration=8.914922803 podStartE2EDuration="27.951761567s" podCreationTimestamp="2026-05-21 11:30:45 +0000 UTC" firstStartedPulling="2026-05-21 11:30:46.950110414 +0000 UTC m=+1.845393939" lastFinishedPulling="2026-05-21 11:31:05.986949172 +0000 UTC m=+20.882232703" observedRunningTime="2026-05-21 11:31:12.949968488 +0000 UTC m=+27.845252033" watchObservedRunningTime="2026-05-21 11:31:12.951761567 +0000 UTC m=+27.847045113" May 21 11:31:13.899551 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:13.899465 2564 generic.go:358] "Generic (PLEG): container finished" podID="bfa04605-9b0c-4c9b-a808-598623c60d2d" containerID="ba6355258daaf59106207a07a48ea44f23e7108c961a048f8ffbce89d5b62ede" exitCode=0 May 21 11:31:13.899551 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:13.899548 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9nh44" event={"ID":"bfa04605-9b0c-4c9b-a808-598623c60d2d","Type":"ContainerDied","Data":"ba6355258daaf59106207a07a48ea44f23e7108c961a048f8ffbce89d5b62ede"} May 21 11:31:14.679819 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:14.679791 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vw6wd" May 21 11:31:14.679980 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:14.679914 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vw6wd" podUID="8316b46b-bb73-4b4e-805b-da0101c1b222" May 21 11:31:14.679980 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:14.679969 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xf5gc" May 21 11:31:14.680113 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:14.680097 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xf5gc" podUID="414e12fe-b9ac-4bbf-8470-ad442215f5cb" May 21 11:31:14.903401 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:14.903365 2564 generic.go:358] "Generic (PLEG): container finished" podID="bfa04605-9b0c-4c9b-a808-598623c60d2d" containerID="8dcb1479a47d6a788e66d3bd47d33d05874c27222abcb65cb3a65b3639add97f" exitCode=0 May 21 11:31:14.903750 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:14.903461 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9nh44" event={"ID":"bfa04605-9b0c-4c9b-a808-598623c60d2d","Type":"ContainerDied","Data":"8dcb1479a47d6a788e66d3bd47d33d05874c27222abcb65cb3a65b3639add97f"} May 21 11:31:16.679713 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:16.679672 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vw6wd" May 21 11:31:16.680402 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:16.679679 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xf5gc" May 21 11:31:16.680402 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:16.679799 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vw6wd" podUID="8316b46b-bb73-4b4e-805b-da0101c1b222" May 21 11:31:16.680402 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:16.679871 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xf5gc" podUID="414e12fe-b9ac-4bbf-8470-ad442215f5cb" May 21 11:31:18.395181 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:18.395145 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/414e12fe-b9ac-4bbf-8470-ad442215f5cb-metrics-certs\") pod \"network-metrics-daemon-xf5gc\" (UID: \"414e12fe-b9ac-4bbf-8470-ad442215f5cb\") " pod="openshift-multus/network-metrics-daemon-xf5gc" May 21 11:31:18.395711 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:18.395295 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 21 11:31:18.395711 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:18.395358 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/414e12fe-b9ac-4bbf-8470-ad442215f5cb-metrics-certs podName:414e12fe-b9ac-4bbf-8470-ad442215f5cb nodeName:}" failed. No retries permitted until 2026-05-21 11:31:50.395339238 +0000 UTC m=+65.290622776 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/414e12fe-b9ac-4bbf-8470-ad442215f5cb-metrics-certs") pod "network-metrics-daemon-xf5gc" (UID: "414e12fe-b9ac-4bbf-8470-ad442215f5cb") : object "openshift-multus"/"metrics-daemon-secret" not registered May 21 11:31:18.496128 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:18.496088 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kvd92\" (UniqueName: \"kubernetes.io/projected/8316b46b-bb73-4b4e-805b-da0101c1b222-kube-api-access-kvd92\") pod \"network-check-target-vw6wd\" (UID: \"8316b46b-bb73-4b4e-805b-da0101c1b222\") " pod="openshift-network-diagnostics/network-check-target-vw6wd" May 21 11:31:18.496405 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:18.496230 2564 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 21 11:31:18.496405 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:18.496259 2564 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 21 11:31:18.496405 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:18.496282 2564 projected.go:194] Error preparing data for projected volume kube-api-access-kvd92 for pod openshift-network-diagnostics/network-check-target-vw6wd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 21 11:31:18.496405 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:18.496342 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8316b46b-bb73-4b4e-805b-da0101c1b222-kube-api-access-kvd92 podName:8316b46b-bb73-4b4e-805b-da0101c1b222 nodeName:}" failed. No retries permitted until 2026-05-21 11:31:50.496322285 +0000 UTC m=+65.391605818 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-kvd92" (UniqueName: "kubernetes.io/projected/8316b46b-bb73-4b4e-805b-da0101c1b222-kube-api-access-kvd92") pod "network-check-target-vw6wd" (UID: "8316b46b-bb73-4b4e-805b-da0101c1b222") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 21 11:31:18.679572 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:18.679493 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vw6wd" May 21 11:31:18.679723 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:18.679506 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xf5gc" May 21 11:31:18.679723 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:18.679615 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vw6wd" podUID="8316b46b-bb73-4b4e-805b-da0101c1b222" May 21 11:31:18.679723 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:18.679714 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xf5gc" podUID="414e12fe-b9ac-4bbf-8470-ad442215f5cb" May 21 11:31:18.949001 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:18.948924 2564 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-213.ec2.internal" event="NodeReady" May 21 11:31:18.949184 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:18.949101 2564 kubelet_node_status.go:550] "Fast updating node status as it just became ready" May 21 11:31:19.004928 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:19.004889 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-s79qs"] May 21 11:31:19.007547 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:19.007524 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-s79qs" May 21 11:31:19.010227 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:19.010205 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-2pfk6"] May 21 11:31:19.010821 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:19.010799 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-q5l2g\"" May 21 11:31:19.011122 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:19.011064 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" May 21 11:31:19.011209 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:19.011065 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" May 21 11:31:19.011880 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:19.011860 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2pfk6" May 21 11:31:19.012296 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:19.012275 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" May 21 11:31:19.015004 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:19.014982 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" May 21 11:31:19.015343 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:19.015326 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" May 21 11:31:19.015899 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:19.015880 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wrdr2\"" May 21 11:31:19.022847 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:19.022820 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-s79qs"] May 21 11:31:19.045552 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:19.045521 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2pfk6"] May 21 11:31:19.101742 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:19.101714 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2z7d\" (UniqueName: \"kubernetes.io/projected/b14cbac4-0b39-404a-a413-a5ccf5fb6472-kube-api-access-r2z7d\") pod \"ingress-canary-s79qs\" (UID: \"b14cbac4-0b39-404a-a413-a5ccf5fb6472\") " pod="openshift-ingress-canary/ingress-canary-s79qs" May 21 11:31:19.101742 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:19.101755 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b14cbac4-0b39-404a-a413-a5ccf5fb6472-cert\") pod \"ingress-canary-s79qs\" (UID: \"b14cbac4-0b39-404a-a413-a5ccf5fb6472\") " pod="openshift-ingress-canary/ingress-canary-s79qs" May 21 11:31:19.102001 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:19.101791 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grshg\" (UniqueName: \"kubernetes.io/projected/2190640c-a631-4c98-8d97-ade39bb1567a-kube-api-access-grshg\") pod \"dns-default-2pfk6\" (UID: \"2190640c-a631-4c98-8d97-ade39bb1567a\") " pod="openshift-dns/dns-default-2pfk6" May 21 11:31:19.102001 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:19.101884 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2190640c-a631-4c98-8d97-ade39bb1567a-tmp-dir\") pod \"dns-default-2pfk6\" (UID: \"2190640c-a631-4c98-8d97-ade39bb1567a\") " pod="openshift-dns/dns-default-2pfk6" May 21 11:31:19.102001 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:19.101922 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2190640c-a631-4c98-8d97-ade39bb1567a-metrics-tls\") pod \"dns-default-2pfk6\" (UID: \"2190640c-a631-4c98-8d97-ade39bb1567a\") " pod="openshift-dns/dns-default-2pfk6" May 21 11:31:19.102001 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:19.101942 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2190640c-a631-4c98-8d97-ade39bb1567a-config-volume\") pod \"dns-default-2pfk6\" (UID: \"2190640c-a631-4c98-8d97-ade39bb1567a\") " pod="openshift-dns/dns-default-2pfk6" May 21 11:31:19.203150 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:19.203034 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-r2z7d\" (UniqueName: \"kubernetes.io/projected/b14cbac4-0b39-404a-a413-a5ccf5fb6472-kube-api-access-r2z7d\") pod \"ingress-canary-s79qs\" (UID: \"b14cbac4-0b39-404a-a413-a5ccf5fb6472\") " pod="openshift-ingress-canary/ingress-canary-s79qs" May 21 11:31:19.203325 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:19.203143 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b14cbac4-0b39-404a-a413-a5ccf5fb6472-cert\") pod \"ingress-canary-s79qs\" (UID: \"b14cbac4-0b39-404a-a413-a5ccf5fb6472\") " pod="openshift-ingress-canary/ingress-canary-s79qs" May 21 11:31:19.203325 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:19.203205 2564 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 21 11:31:19.203325 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:19.203233 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-grshg\" (UniqueName: \"kubernetes.io/projected/2190640c-a631-4c98-8d97-ade39bb1567a-kube-api-access-grshg\") pod \"dns-default-2pfk6\" (UID: \"2190640c-a631-4c98-8d97-ade39bb1567a\") " pod="openshift-dns/dns-default-2pfk6" May 21 11:31:19.203325 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:19.203284 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b14cbac4-0b39-404a-a413-a5ccf5fb6472-cert podName:b14cbac4-0b39-404a-a413-a5ccf5fb6472 nodeName:}" failed. No retries permitted until 2026-05-21 11:31:19.703263305 +0000 UTC m=+34.598546852 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/b14cbac4-0b39-404a-a413-a5ccf5fb6472-cert") pod "ingress-canary-s79qs" (UID: "b14cbac4-0b39-404a-a413-a5ccf5fb6472") : secret "canary-serving-cert" not found May 21 11:31:19.203530 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:19.203326 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2190640c-a631-4c98-8d97-ade39bb1567a-tmp-dir\") pod \"dns-default-2pfk6\" (UID: \"2190640c-a631-4c98-8d97-ade39bb1567a\") " pod="openshift-dns/dns-default-2pfk6" May 21 11:31:19.203530 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:19.203371 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2190640c-a631-4c98-8d97-ade39bb1567a-metrics-tls\") pod \"dns-default-2pfk6\" (UID: \"2190640c-a631-4c98-8d97-ade39bb1567a\") " pod="openshift-dns/dns-default-2pfk6" May 21 11:31:19.203530 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:19.203401 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2190640c-a631-4c98-8d97-ade39bb1567a-config-volume\") pod \"dns-default-2pfk6\" (UID: \"2190640c-a631-4c98-8d97-ade39bb1567a\") " pod="openshift-dns/dns-default-2pfk6" May 21 11:31:19.203530 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:19.203506 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 21 11:31:19.203669 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:19.203568 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2190640c-a631-4c98-8d97-ade39bb1567a-metrics-tls podName:2190640c-a631-4c98-8d97-ade39bb1567a nodeName:}" failed. No retries permitted until 2026-05-21 11:31:19.703551177 +0000 UTC m=+34.598834713 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2190640c-a631-4c98-8d97-ade39bb1567a-metrics-tls") pod "dns-default-2pfk6" (UID: "2190640c-a631-4c98-8d97-ade39bb1567a") : secret "dns-default-metrics-tls" not found May 21 11:31:19.204253 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:19.204228 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2190640c-a631-4c98-8d97-ade39bb1567a-tmp-dir\") pod \"dns-default-2pfk6\" (UID: \"2190640c-a631-4c98-8d97-ade39bb1567a\") " pod="openshift-dns/dns-default-2pfk6" May 21 11:31:19.204435 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:19.204419 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2190640c-a631-4c98-8d97-ade39bb1567a-config-volume\") pod \"dns-default-2pfk6\" (UID: \"2190640c-a631-4c98-8d97-ade39bb1567a\") " pod="openshift-dns/dns-default-2pfk6" May 21 11:31:19.215056 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:19.215030 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-grshg\" (UniqueName: \"kubernetes.io/projected/2190640c-a631-4c98-8d97-ade39bb1567a-kube-api-access-grshg\") pod \"dns-default-2pfk6\" (UID: \"2190640c-a631-4c98-8d97-ade39bb1567a\") " pod="openshift-dns/dns-default-2pfk6" May 21 11:31:19.215190 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:19.215072 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2z7d\" (UniqueName: \"kubernetes.io/projected/b14cbac4-0b39-404a-a413-a5ccf5fb6472-kube-api-access-r2z7d\") pod \"ingress-canary-s79qs\" (UID: \"b14cbac4-0b39-404a-a413-a5ccf5fb6472\") " pod="openshift-ingress-canary/ingress-canary-s79qs" May 21 11:31:19.707378 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:19.707344 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2190640c-a631-4c98-8d97-ade39bb1567a-metrics-tls\") pod \"dns-default-2pfk6\" (UID: \"2190640c-a631-4c98-8d97-ade39bb1567a\") " pod="openshift-dns/dns-default-2pfk6" May 21 11:31:19.707898 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:19.707408 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b14cbac4-0b39-404a-a413-a5ccf5fb6472-cert\") pod \"ingress-canary-s79qs\" (UID: \"b14cbac4-0b39-404a-a413-a5ccf5fb6472\") " pod="openshift-ingress-canary/ingress-canary-s79qs" May 21 11:31:19.707898 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:19.707493 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 21 11:31:19.707898 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:19.707521 2564 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 21 11:31:19.707898 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:19.707566 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b14cbac4-0b39-404a-a413-a5ccf5fb6472-cert podName:b14cbac4-0b39-404a-a413-a5ccf5fb6472 nodeName:}" failed. No retries permitted until 2026-05-21 11:31:20.707550061 +0000 UTC m=+35.602833598 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/b14cbac4-0b39-404a-a413-a5ccf5fb6472-cert") pod "ingress-canary-s79qs" (UID: "b14cbac4-0b39-404a-a413-a5ccf5fb6472") : secret "canary-serving-cert" not found May 21 11:31:19.707898 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:19.707582 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2190640c-a631-4c98-8d97-ade39bb1567a-metrics-tls podName:2190640c-a631-4c98-8d97-ade39bb1567a nodeName:}" failed. No retries permitted until 2026-05-21 11:31:20.707574269 +0000 UTC m=+35.602857791 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2190640c-a631-4c98-8d97-ade39bb1567a-metrics-tls") pod "dns-default-2pfk6" (UID: "2190640c-a631-4c98-8d97-ade39bb1567a") : secret "dns-default-metrics-tls" not found May 21 11:31:20.679877 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:20.679840 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xf5gc" May 21 11:31:20.680063 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:20.679841 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vw6wd" May 21 11:31:20.685158 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:20.685129 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" May 21 11:31:20.685303 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:20.685168 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" May 21 11:31:20.685303 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:20.685198 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tr4cg\"" May 21 11:31:20.685303 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:20.685220 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-cdxc2\"" May 21 11:31:20.685303 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:20.685170 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" May 21 11:31:20.713607 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:20.713579 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2190640c-a631-4c98-8d97-ade39bb1567a-metrics-tls\") pod \"dns-default-2pfk6\" (UID: \"2190640c-a631-4c98-8d97-ade39bb1567a\") " pod="openshift-dns/dns-default-2pfk6" May 21 11:31:20.713917 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:20.713619 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b14cbac4-0b39-404a-a413-a5ccf5fb6472-cert\") pod \"ingress-canary-s79qs\" (UID: \"b14cbac4-0b39-404a-a413-a5ccf5fb6472\") " pod="openshift-ingress-canary/ingress-canary-s79qs" May 21 11:31:20.713917 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:20.713715 2564 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 21 11:31:20.713917 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:20.713726 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 21 11:31:20.713917 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:20.713765 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b14cbac4-0b39-404a-a413-a5ccf5fb6472-cert podName:b14cbac4-0b39-404a-a413-a5ccf5fb6472 nodeName:}" failed. No retries permitted until 2026-05-21 11:31:22.713750341 +0000 UTC m=+37.609033864 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/b14cbac4-0b39-404a-a413-a5ccf5fb6472-cert") pod "ingress-canary-s79qs" (UID: "b14cbac4-0b39-404a-a413-a5ccf5fb6472") : secret "canary-serving-cert" not found May 21 11:31:20.713917 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:20.713785 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2190640c-a631-4c98-8d97-ade39bb1567a-metrics-tls podName:2190640c-a631-4c98-8d97-ade39bb1567a nodeName:}" failed. No retries permitted until 2026-05-21 11:31:22.713771054 +0000 UTC m=+37.609054577 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2190640c-a631-4c98-8d97-ade39bb1567a-metrics-tls") pod "dns-default-2pfk6" (UID: "2190640c-a631-4c98-8d97-ade39bb1567a") : secret "dns-default-metrics-tls" not found May 21 11:31:21.920967 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:21.920935 2564 generic.go:358] "Generic (PLEG): container finished" podID="bfa04605-9b0c-4c9b-a808-598623c60d2d" containerID="11528ae8f415c0842c39d8df9efd488a1081df53b014c41363b934a35d6e9da7" exitCode=0 May 21 11:31:21.921441 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:21.920979 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9nh44" event={"ID":"bfa04605-9b0c-4c9b-a808-598623c60d2d","Type":"ContainerDied","Data":"11528ae8f415c0842c39d8df9efd488a1081df53b014c41363b934a35d6e9da7"} May 21 11:31:22.729013 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:22.728928 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2190640c-a631-4c98-8d97-ade39bb1567a-metrics-tls\") pod \"dns-default-2pfk6\" (UID: \"2190640c-a631-4c98-8d97-ade39bb1567a\") " pod="openshift-dns/dns-default-2pfk6" May 21 11:31:22.729013 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:22.728979 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b14cbac4-0b39-404a-a413-a5ccf5fb6472-cert\") pod \"ingress-canary-s79qs\" (UID: \"b14cbac4-0b39-404a-a413-a5ccf5fb6472\") " pod="openshift-ingress-canary/ingress-canary-s79qs" May 21 11:31:22.729230 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:22.729099 2564 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 21 11:31:22.729230 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:22.729101 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 21 11:31:22.729230 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:22.729154 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b14cbac4-0b39-404a-a413-a5ccf5fb6472-cert podName:b14cbac4-0b39-404a-a413-a5ccf5fb6472 nodeName:}" failed. No retries permitted until 2026-05-21 11:31:26.729140481 +0000 UTC m=+41.624424003 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/b14cbac4-0b39-404a-a413-a5ccf5fb6472-cert") pod "ingress-canary-s79qs" (UID: "b14cbac4-0b39-404a-a413-a5ccf5fb6472") : secret "canary-serving-cert" not found May 21 11:31:22.729230 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:22.729167 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2190640c-a631-4c98-8d97-ade39bb1567a-metrics-tls podName:2190640c-a631-4c98-8d97-ade39bb1567a nodeName:}" failed. No retries permitted until 2026-05-21 11:31:26.729160614 +0000 UTC m=+41.624444137 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2190640c-a631-4c98-8d97-ade39bb1567a-metrics-tls") pod "dns-default-2pfk6" (UID: "2190640c-a631-4c98-8d97-ade39bb1567a") : secret "dns-default-metrics-tls" not found May 21 11:31:22.926582 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:22.926545 2564 generic.go:358] "Generic (PLEG): container finished" podID="bfa04605-9b0c-4c9b-a808-598623c60d2d" containerID="ab762398dbb69c8afdab235db66cddd556fb52d8b54e951d00017de50b304396" exitCode=0 May 21 11:31:22.926957 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:22.926599 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9nh44" event={"ID":"bfa04605-9b0c-4c9b-a808-598623c60d2d","Type":"ContainerDied","Data":"ab762398dbb69c8afdab235db66cddd556fb52d8b54e951d00017de50b304396"} May 21 11:31:23.930802 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:23.930770 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9nh44" event={"ID":"bfa04605-9b0c-4c9b-a808-598623c60d2d","Type":"ContainerStarted","Data":"4bbbd6b3df52669fc5964518519eaed9f7fc30328746aed8b619fa5f8f302961"} May 21 11:31:23.956694 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:23.956652 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-9nh44" podStartSLOduration=4.734962884 podStartE2EDuration="38.956638095s" podCreationTimestamp="2026-05-21 11:30:45 +0000 UTC" firstStartedPulling="2026-05-21 11:30:46.906787766 +0000 UTC m=+1.802071293" lastFinishedPulling="2026-05-21 11:31:21.128462981 +0000 UTC m=+36.023746504" observedRunningTime="2026-05-21 11:31:23.95633536 +0000 UTC m=+38.851618919" watchObservedRunningTime="2026-05-21 11:31:23.956638095 +0000 UTC m=+38.851921639" May 21 11:31:26.757496 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:26.757441 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b14cbac4-0b39-404a-a413-a5ccf5fb6472-cert\") pod \"ingress-canary-s79qs\" (UID: \"b14cbac4-0b39-404a-a413-a5ccf5fb6472\") " pod="openshift-ingress-canary/ingress-canary-s79qs" May 21 11:31:26.758009 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:26.757549 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2190640c-a631-4c98-8d97-ade39bb1567a-metrics-tls\") pod \"dns-default-2pfk6\" (UID: \"2190640c-a631-4c98-8d97-ade39bb1567a\") " pod="openshift-dns/dns-default-2pfk6" May 21 11:31:26.758009 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:26.757593 2564 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 21 11:31:26.758009 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:26.757664 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 21 11:31:26.758009 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:26.757666 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b14cbac4-0b39-404a-a413-a5ccf5fb6472-cert podName:b14cbac4-0b39-404a-a413-a5ccf5fb6472 nodeName:}" failed. No retries permitted until 2026-05-21 11:31:34.757651001 +0000 UTC m=+49.652934528 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/b14cbac4-0b39-404a-a413-a5ccf5fb6472-cert") pod "ingress-canary-s79qs" (UID: "b14cbac4-0b39-404a-a413-a5ccf5fb6472") : secret "canary-serving-cert" not found May 21 11:31:26.758009 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:26.757722 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2190640c-a631-4c98-8d97-ade39bb1567a-metrics-tls podName:2190640c-a631-4c98-8d97-ade39bb1567a nodeName:}" failed. No retries permitted until 2026-05-21 11:31:34.757706824 +0000 UTC m=+49.652990348 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2190640c-a631-4c98-8d97-ade39bb1567a-metrics-tls") pod "dns-default-2pfk6" (UID: "2190640c-a631-4c98-8d97-ade39bb1567a") : secret "dns-default-metrics-tls" not found May 21 11:31:34.814340 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:34.814295 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b14cbac4-0b39-404a-a413-a5ccf5fb6472-cert\") pod \"ingress-canary-s79qs\" (UID: \"b14cbac4-0b39-404a-a413-a5ccf5fb6472\") " pod="openshift-ingress-canary/ingress-canary-s79qs" May 21 11:31:34.814791 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:34.814385 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2190640c-a631-4c98-8d97-ade39bb1567a-metrics-tls\") pod \"dns-default-2pfk6\" (UID: \"2190640c-a631-4c98-8d97-ade39bb1567a\") " pod="openshift-dns/dns-default-2pfk6" May 21 11:31:34.814791 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:34.814467 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 21 11:31:34.814791 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:34.814473 2564 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 21 11:31:34.814791 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:34.814519 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2190640c-a631-4c98-8d97-ade39bb1567a-metrics-tls podName:2190640c-a631-4c98-8d97-ade39bb1567a nodeName:}" failed. No retries permitted until 2026-05-21 11:31:50.814505874 +0000 UTC m=+65.709789397 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2190640c-a631-4c98-8d97-ade39bb1567a-metrics-tls") pod "dns-default-2pfk6" (UID: "2190640c-a631-4c98-8d97-ade39bb1567a") : secret "dns-default-metrics-tls" not found May 21 11:31:34.814791 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:34.814542 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b14cbac4-0b39-404a-a413-a5ccf5fb6472-cert podName:b14cbac4-0b39-404a-a413-a5ccf5fb6472 nodeName:}" failed. No retries permitted until 2026-05-21 11:31:50.814524988 +0000 UTC m=+65.709808517 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/b14cbac4-0b39-404a-a413-a5ccf5fb6472-cert") pod "ingress-canary-s79qs" (UID: "b14cbac4-0b39-404a-a413-a5ccf5fb6472") : secret "canary-serving-cert" not found May 21 11:31:44.914589 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:44.914560 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sr2wt" May 21 11:31:50.426380 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:50.426341 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/414e12fe-b9ac-4bbf-8470-ad442215f5cb-metrics-certs\") pod \"network-metrics-daemon-xf5gc\" (UID: \"414e12fe-b9ac-4bbf-8470-ad442215f5cb\") " pod="openshift-multus/network-metrics-daemon-xf5gc" May 21 11:31:50.429888 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:50.429864 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" May 21 11:31:50.436554 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:50.436529 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found May 21 11:31:50.436637 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:50.436609 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/414e12fe-b9ac-4bbf-8470-ad442215f5cb-metrics-certs podName:414e12fe-b9ac-4bbf-8470-ad442215f5cb nodeName:}" failed. No retries permitted until 2026-05-21 11:32:54.436592206 +0000 UTC m=+129.331875729 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/414e12fe-b9ac-4bbf-8470-ad442215f5cb-metrics-certs") pod "network-metrics-daemon-xf5gc" (UID: "414e12fe-b9ac-4bbf-8470-ad442215f5cb") : secret "metrics-daemon-secret" not found May 21 11:31:50.527160 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:50.527127 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kvd92\" (UniqueName: \"kubernetes.io/projected/8316b46b-bb73-4b4e-805b-da0101c1b222-kube-api-access-kvd92\") pod \"network-check-target-vw6wd\" (UID: \"8316b46b-bb73-4b4e-805b-da0101c1b222\") " pod="openshift-network-diagnostics/network-check-target-vw6wd" May 21 11:31:50.530450 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:50.530433 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" May 21 11:31:50.540561 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:50.540542 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" May 21 11:31:50.551245 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:50.551212 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvd92\" (UniqueName: \"kubernetes.io/projected/8316b46b-bb73-4b4e-805b-da0101c1b222-kube-api-access-kvd92\") pod \"network-check-target-vw6wd\" (UID: \"8316b46b-bb73-4b4e-805b-da0101c1b222\") " pod="openshift-network-diagnostics/network-check-target-vw6wd" May 21 11:31:50.701525 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:50.701444 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-cdxc2\"" May 21 11:31:50.708139 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:50.708122 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vw6wd" May 21 11:31:50.830201 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:50.830150 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2190640c-a631-4c98-8d97-ade39bb1567a-metrics-tls\") pod \"dns-default-2pfk6\" (UID: \"2190640c-a631-4c98-8d97-ade39bb1567a\") " pod="openshift-dns/dns-default-2pfk6" May 21 11:31:50.830364 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:50.830216 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b14cbac4-0b39-404a-a413-a5ccf5fb6472-cert\") pod \"ingress-canary-s79qs\" (UID: \"b14cbac4-0b39-404a-a413-a5ccf5fb6472\") " pod="openshift-ingress-canary/ingress-canary-s79qs" May 21 11:31:50.830364 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:50.830346 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 21 11:31:50.830364 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:50.830356 2564 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 21 11:31:50.830553 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:50.830414 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b14cbac4-0b39-404a-a413-a5ccf5fb6472-cert podName:b14cbac4-0b39-404a-a413-a5ccf5fb6472 nodeName:}" failed. No retries permitted until 2026-05-21 11:32:22.830399716 +0000 UTC m=+97.725683239 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/b14cbac4-0b39-404a-a413-a5ccf5fb6472-cert") pod "ingress-canary-s79qs" (UID: "b14cbac4-0b39-404a-a413-a5ccf5fb6472") : secret "canary-serving-cert" not found May 21 11:31:50.830553 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:31:50.830427 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2190640c-a631-4c98-8d97-ade39bb1567a-metrics-tls podName:2190640c-a631-4c98-8d97-ade39bb1567a nodeName:}" failed. No retries permitted until 2026-05-21 11:32:22.830421296 +0000 UTC m=+97.725704819 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2190640c-a631-4c98-8d97-ade39bb1567a-metrics-tls") pod "dns-default-2pfk6" (UID: "2190640c-a631-4c98-8d97-ade39bb1567a") : secret "dns-default-metrics-tls" not found May 21 11:31:50.875403 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:50.875376 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vw6wd"] May 21 11:31:50.878775 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:31:50.878745 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8316b46b_bb73_4b4e_805b_da0101c1b222.slice/crio-a24c4bbf364de89a48f3f44e19000237b892118115b10c560730d9cef3d5977b WatchSource:0}: Error finding container a24c4bbf364de89a48f3f44e19000237b892118115b10c560730d9cef3d5977b: Status 404 returned error can't find the container with id a24c4bbf364de89a48f3f44e19000237b892118115b10c560730d9cef3d5977b May 21 11:31:50.986285 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:50.986199 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vw6wd" event={"ID":"8316b46b-bb73-4b4e-805b-da0101c1b222","Type":"ContainerStarted","Data":"a24c4bbf364de89a48f3f44e19000237b892118115b10c560730d9cef3d5977b"} May 21 11:31:53.993325 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:53.993289 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vw6wd" event={"ID":"8316b46b-bb73-4b4e-805b-da0101c1b222","Type":"ContainerStarted","Data":"319160b491598f1789f05b1a75a02f8ace0635a1d7358f878339ba57d54469b2"} May 21 11:31:53.993674 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:53.993425 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-vw6wd" May 21 11:31:54.013491 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:31:54.013445 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-vw6wd" podStartSLOduration=66.011761974 podStartE2EDuration="1m9.01343243s" podCreationTimestamp="2026-05-21 11:30:45 +0000 UTC" firstStartedPulling="2026-05-21 11:31:50.88078829 +0000 UTC m=+65.776071816" lastFinishedPulling="2026-05-21 11:31:53.88245874 +0000 UTC m=+68.777742272" observedRunningTime="2026-05-21 11:31:54.012885591 +0000 UTC m=+68.908169149" watchObservedRunningTime="2026-05-21 11:31:54.01343243 +0000 UTC m=+68.908715968" May 21 11:32:22.857997 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:22.857941 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b14cbac4-0b39-404a-a413-a5ccf5fb6472-cert\") pod \"ingress-canary-s79qs\" (UID: \"b14cbac4-0b39-404a-a413-a5ccf5fb6472\") " pod="openshift-ingress-canary/ingress-canary-s79qs" May 21 11:32:22.858476 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:22.858026 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2190640c-a631-4c98-8d97-ade39bb1567a-metrics-tls\") pod \"dns-default-2pfk6\" (UID: \"2190640c-a631-4c98-8d97-ade39bb1567a\") " pod="openshift-dns/dns-default-2pfk6" May 21 11:32:22.858476 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:32:22.858117 2564 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 21 11:32:22.858476 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:32:22.858125 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 21 11:32:22.858476 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:32:22.858198 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2190640c-a631-4c98-8d97-ade39bb1567a-metrics-tls podName:2190640c-a631-4c98-8d97-ade39bb1567a nodeName:}" failed. No retries permitted until 2026-05-21 11:33:26.858184151 +0000 UTC m=+161.753467674 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2190640c-a631-4c98-8d97-ade39bb1567a-metrics-tls") pod "dns-default-2pfk6" (UID: "2190640c-a631-4c98-8d97-ade39bb1567a") : secret "dns-default-metrics-tls" not found May 21 11:32:22.858476 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:32:22.858212 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b14cbac4-0b39-404a-a413-a5ccf5fb6472-cert podName:b14cbac4-0b39-404a-a413-a5ccf5fb6472 nodeName:}" failed. No retries permitted until 2026-05-21 11:33:26.858205518 +0000 UTC m=+161.753489041 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/b14cbac4-0b39-404a-a413-a5ccf5fb6472-cert") pod "ingress-canary-s79qs" (UID: "b14cbac4-0b39-404a-a413-a5ccf5fb6472") : secret "canary-serving-cert" not found May 21 11:32:24.998201 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:24.998150 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-vw6wd" May 21 11:32:48.964448 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:48.964410 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-xdnz5"] May 21 11:32:48.966985 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:48.966962 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-76dff48d59-6g9xk"] May 21 11:32:48.967152 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:48.967133 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-xdnz5" May 21 11:32:48.968605 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:48.968588 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-76dff48d59-6g9xk" May 21 11:32:48.970105 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:48.970064 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" May 21 11:32:48.970105 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:48.970102 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" May 21 11:32:48.970276 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:48.970072 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" May 21 11:32:48.970276 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:48.970252 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-kgrw7\"" May 21 11:32:48.971892 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:48.971874 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" May 21 11:32:48.972033 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:48.971999 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" May 21 11:32:48.972129 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:48.972053 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" May 21 11:32:48.972129 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:48.972013 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" May 21 11:32:48.972243 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:48.971941 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" May 21 11:32:48.972243 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:48.972014 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-7bvxx\"" May 21 11:32:48.977595 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:48.977577 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" May 21 11:32:48.977773 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:48.977710 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-xdnz5"] May 21 11:32:48.980755 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:48.980731 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-76dff48d59-6g9xk"] May 21 11:32:49.038550 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.038504 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a74a4dc1-ca52-40f4-9cb2-0c925a08ecfd-serving-cert\") pod \"kube-storage-version-migrator-operator-565644f49-xdnz5\" (UID: \"a74a4dc1-ca52-40f4-9cb2-0c925a08ecfd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-xdnz5" May 21 11:32:49.038550 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.038554 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/be463222-9e99-47cb-bf6b-525913ca9e29-config\") pod \"console-operator-76dff48d59-6g9xk\" (UID: \"be463222-9e99-47cb-bf6b-525913ca9e29\") " pod="openshift-console-operator/console-operator-76dff48d59-6g9xk" May 21 11:32:49.038779 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.038574 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/be463222-9e99-47cb-bf6b-525913ca9e29-trusted-ca\") pod \"console-operator-76dff48d59-6g9xk\" (UID: \"be463222-9e99-47cb-bf6b-525913ca9e29\") " pod="openshift-console-operator/console-operator-76dff48d59-6g9xk" May 21 11:32:49.038779 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.038593 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlt26\" (UniqueName: \"kubernetes.io/projected/be463222-9e99-47cb-bf6b-525913ca9e29-kube-api-access-qlt26\") pod \"console-operator-76dff48d59-6g9xk\" (UID: \"be463222-9e99-47cb-bf6b-525913ca9e29\") " pod="openshift-console-operator/console-operator-76dff48d59-6g9xk" May 21 11:32:49.038779 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.038620 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/be463222-9e99-47cb-bf6b-525913ca9e29-serving-cert\") pod \"console-operator-76dff48d59-6g9xk\" (UID: \"be463222-9e99-47cb-bf6b-525913ca9e29\") " pod="openshift-console-operator/console-operator-76dff48d59-6g9xk" May 21 11:32:49.038779 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.038653 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbsq6\" (UniqueName: \"kubernetes.io/projected/a74a4dc1-ca52-40f4-9cb2-0c925a08ecfd-kube-api-access-cbsq6\") pod \"kube-storage-version-migrator-operator-565644f49-xdnz5\" (UID: \"a74a4dc1-ca52-40f4-9cb2-0c925a08ecfd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-xdnz5" May 21 11:32:49.038779 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.038722 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a74a4dc1-ca52-40f4-9cb2-0c925a08ecfd-config\") pod \"kube-storage-version-migrator-operator-565644f49-xdnz5\" (UID: \"a74a4dc1-ca52-40f4-9cb2-0c925a08ecfd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-xdnz5" May 21 11:32:49.070505 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.070473 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-697587599-vv4gd"] May 21 11:32:49.072455 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.072440 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-vv4gd" May 21 11:32:49.075517 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.075493 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-78bdfb888c-49stn"] May 21 11:32:49.076035 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.076002 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" May 21 11:32:49.076168 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.076108 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" May 21 11:32:49.076384 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.076366 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-9sx8t\"" May 21 11:32:49.076457 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.076444 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" May 21 11:32:49.077342 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.077328 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-b7f67965d-272k2"] May 21 11:32:49.077484 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.077469 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-78bdfb888c-49stn" May 21 11:32:49.079139 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.079119 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-b7f67965d-272k2" May 21 11:32:49.082046 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.082024 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" May 21 11:32:49.082244 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.082228 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" May 21 11:32:49.082327 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.082296 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-lkj99\"" May 21 11:32:49.082466 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.082452 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" May 21 11:32:49.082524 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.082510 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" May 21 11:32:49.082829 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.082814 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" May 21 11:32:49.082879 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.082831 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" May 21 11:32:49.082934 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.082917 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" May 21 11:32:49.083000 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.082927 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-gz2j6\"" May 21 11:32:49.083556 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.083543 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" May 21 11:32:49.083601 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.083567 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" May 21 11:32:49.083601 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.083574 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" May 21 11:32:49.088781 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.088763 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-697587599-vv4gd"] May 21 11:32:49.090844 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.090823 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" May 21 11:32:49.098715 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.098693 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-78bdfb888c-49stn"] May 21 11:32:49.108377 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.108356 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-b7f67965d-272k2"] May 21 11:32:49.139357 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.139333 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a74a4dc1-ca52-40f4-9cb2-0c925a08ecfd-config\") pod \"kube-storage-version-migrator-operator-565644f49-xdnz5\" (UID: \"a74a4dc1-ca52-40f4-9cb2-0c925a08ecfd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-xdnz5" May 21 11:32:49.139491 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.139386 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4540e5a0-576b-4722-acfc-a2cec978f9e3-service-ca-bundle\") pod \"router-default-b7f67965d-272k2\" (UID: \"4540e5a0-576b-4722-acfc-a2cec978f9e3\") " pod="openshift-ingress/router-default-b7f67965d-272k2" May 21 11:32:49.139573 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.139512 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a74a4dc1-ca52-40f4-9cb2-0c925a08ecfd-serving-cert\") pod \"kube-storage-version-migrator-operator-565644f49-xdnz5\" (UID: \"a74a4dc1-ca52-40f4-9cb2-0c925a08ecfd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-xdnz5" May 21 11:32:49.139573 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.139543 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/89aba95e-fa54-4fba-bf2b-0e3569017e24-samples-operator-tls\") pod \"cluster-samples-operator-697587599-vv4gd\" (UID: \"89aba95e-fa54-4fba-bf2b-0e3569017e24\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-vv4gd" May 21 11:32:49.139671 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.139576 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/4540e5a0-576b-4722-acfc-a2cec978f9e3-default-certificate\") pod \"router-default-b7f67965d-272k2\" (UID: \"4540e5a0-576b-4722-acfc-a2cec978f9e3\") " pod="openshift-ingress/router-default-b7f67965d-272k2" May 21 11:32:49.139671 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.139605 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4540e5a0-576b-4722-acfc-a2cec978f9e3-metrics-certs\") pod \"router-default-b7f67965d-272k2\" (UID: \"4540e5a0-576b-4722-acfc-a2cec978f9e3\") " pod="openshift-ingress/router-default-b7f67965d-272k2" May 21 11:32:49.139760 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.139661 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/be463222-9e99-47cb-bf6b-525913ca9e29-config\") pod \"console-operator-76dff48d59-6g9xk\" (UID: \"be463222-9e99-47cb-bf6b-525913ca9e29\") " pod="openshift-console-operator/console-operator-76dff48d59-6g9xk" May 21 11:32:49.139760 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.139706 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/be463222-9e99-47cb-bf6b-525913ca9e29-trusted-ca\") pod \"console-operator-76dff48d59-6g9xk\" (UID: \"be463222-9e99-47cb-bf6b-525913ca9e29\") " pod="openshift-console-operator/console-operator-76dff48d59-6g9xk" May 21 11:32:49.139760 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.139750 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qlt26\" (UniqueName: \"kubernetes.io/projected/be463222-9e99-47cb-bf6b-525913ca9e29-kube-api-access-qlt26\") pod \"console-operator-76dff48d59-6g9xk\" (UID: \"be463222-9e99-47cb-bf6b-525913ca9e29\") " pod="openshift-console-operator/console-operator-76dff48d59-6g9xk" May 21 11:32:49.139890 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.139780 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f797c9ec-815c-4e1a-aa83-9173927d8593-service-ca-bundle\") pod \"insights-operator-78bdfb888c-49stn\" (UID: \"f797c9ec-815c-4e1a-aa83-9173927d8593\") " pod="openshift-insights/insights-operator-78bdfb888c-49stn" May 21 11:32:49.139890 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.139807 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpnjj\" (UniqueName: \"kubernetes.io/projected/4540e5a0-576b-4722-acfc-a2cec978f9e3-kube-api-access-wpnjj\") pod \"router-default-b7f67965d-272k2\" (UID: \"4540e5a0-576b-4722-acfc-a2cec978f9e3\") " pod="openshift-ingress/router-default-b7f67965d-272k2" May 21 11:32:49.139890 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.139849 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/be463222-9e99-47cb-bf6b-525913ca9e29-serving-cert\") pod \"console-operator-76dff48d59-6g9xk\" (UID: \"be463222-9e99-47cb-bf6b-525913ca9e29\") " pod="openshift-console-operator/console-operator-76dff48d59-6g9xk" May 21 11:32:49.139890 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.139884 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f797c9ec-815c-4e1a-aa83-9173927d8593-trusted-ca-bundle\") pod \"insights-operator-78bdfb888c-49stn\" (UID: \"f797c9ec-815c-4e1a-aa83-9173927d8593\") " pod="openshift-insights/insights-operator-78bdfb888c-49stn" May 21 11:32:49.140167 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.139916 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2q74p\" (UniqueName: \"kubernetes.io/projected/89aba95e-fa54-4fba-bf2b-0e3569017e24-kube-api-access-2q74p\") pod \"cluster-samples-operator-697587599-vv4gd\" (UID: \"89aba95e-fa54-4fba-bf2b-0e3569017e24\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-vv4gd" May 21 11:32:49.140167 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.139936 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a74a4dc1-ca52-40f4-9cb2-0c925a08ecfd-config\") pod \"kube-storage-version-migrator-operator-565644f49-xdnz5\" (UID: \"a74a4dc1-ca52-40f4-9cb2-0c925a08ecfd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-xdnz5" May 21 11:32:49.140167 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.139949 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f797c9ec-815c-4e1a-aa83-9173927d8593-tmp\") pod \"insights-operator-78bdfb888c-49stn\" (UID: \"f797c9ec-815c-4e1a-aa83-9173927d8593\") " pod="openshift-insights/insights-operator-78bdfb888c-49stn" May 21 11:32:49.140167 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.139974 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/f797c9ec-815c-4e1a-aa83-9173927d8593-snapshots\") pod \"insights-operator-78bdfb888c-49stn\" (UID: \"f797c9ec-815c-4e1a-aa83-9173927d8593\") " pod="openshift-insights/insights-operator-78bdfb888c-49stn" May 21 11:32:49.140167 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.140015 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f797c9ec-815c-4e1a-aa83-9173927d8593-serving-cert\") pod \"insights-operator-78bdfb888c-49stn\" (UID: \"f797c9ec-815c-4e1a-aa83-9173927d8593\") " pod="openshift-insights/insights-operator-78bdfb888c-49stn" May 21 11:32:49.140167 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.140060 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cbsq6\" (UniqueName: \"kubernetes.io/projected/a74a4dc1-ca52-40f4-9cb2-0c925a08ecfd-kube-api-access-cbsq6\") pod \"kube-storage-version-migrator-operator-565644f49-xdnz5\" (UID: \"a74a4dc1-ca52-40f4-9cb2-0c925a08ecfd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-xdnz5" May 21 11:32:49.140167 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.140121 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dstn4\" (UniqueName: \"kubernetes.io/projected/f797c9ec-815c-4e1a-aa83-9173927d8593-kube-api-access-dstn4\") pod \"insights-operator-78bdfb888c-49stn\" (UID: \"f797c9ec-815c-4e1a-aa83-9173927d8593\") " pod="openshift-insights/insights-operator-78bdfb888c-49stn" May 21 11:32:49.140167 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.140160 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/4540e5a0-576b-4722-acfc-a2cec978f9e3-stats-auth\") pod \"router-default-b7f67965d-272k2\" (UID: \"4540e5a0-576b-4722-acfc-a2cec978f9e3\") " pod="openshift-ingress/router-default-b7f67965d-272k2" May 21 11:32:49.140618 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.140374 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/be463222-9e99-47cb-bf6b-525913ca9e29-config\") pod \"console-operator-76dff48d59-6g9xk\" (UID: \"be463222-9e99-47cb-bf6b-525913ca9e29\") " pod="openshift-console-operator/console-operator-76dff48d59-6g9xk" May 21 11:32:49.141046 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.141024 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/be463222-9e99-47cb-bf6b-525913ca9e29-trusted-ca\") pod \"console-operator-76dff48d59-6g9xk\" (UID: \"be463222-9e99-47cb-bf6b-525913ca9e29\") " pod="openshift-console-operator/console-operator-76dff48d59-6g9xk" May 21 11:32:49.142006 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.141988 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a74a4dc1-ca52-40f4-9cb2-0c925a08ecfd-serving-cert\") pod \"kube-storage-version-migrator-operator-565644f49-xdnz5\" (UID: \"a74a4dc1-ca52-40f4-9cb2-0c925a08ecfd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-xdnz5" May 21 11:32:49.142393 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.142375 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/be463222-9e99-47cb-bf6b-525913ca9e29-serving-cert\") pod \"console-operator-76dff48d59-6g9xk\" (UID: \"be463222-9e99-47cb-bf6b-525913ca9e29\") " pod="openshift-console-operator/console-operator-76dff48d59-6g9xk" May 21 11:32:49.148937 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.148910 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlt26\" (UniqueName: \"kubernetes.io/projected/be463222-9e99-47cb-bf6b-525913ca9e29-kube-api-access-qlt26\") pod \"console-operator-76dff48d59-6g9xk\" (UID: \"be463222-9e99-47cb-bf6b-525913ca9e29\") " pod="openshift-console-operator/console-operator-76dff48d59-6g9xk" May 21 11:32:49.149208 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.149186 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbsq6\" (UniqueName: \"kubernetes.io/projected/a74a4dc1-ca52-40f4-9cb2-0c925a08ecfd-kube-api-access-cbsq6\") pod \"kube-storage-version-migrator-operator-565644f49-xdnz5\" (UID: \"a74a4dc1-ca52-40f4-9cb2-0c925a08ecfd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-xdnz5" May 21 11:32:49.240886 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.240798 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/4540e5a0-576b-4722-acfc-a2cec978f9e3-default-certificate\") pod \"router-default-b7f67965d-272k2\" (UID: \"4540e5a0-576b-4722-acfc-a2cec978f9e3\") " pod="openshift-ingress/router-default-b7f67965d-272k2" May 21 11:32:49.240886 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.240836 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4540e5a0-576b-4722-acfc-a2cec978f9e3-metrics-certs\") pod \"router-default-b7f67965d-272k2\" (UID: \"4540e5a0-576b-4722-acfc-a2cec978f9e3\") " pod="openshift-ingress/router-default-b7f67965d-272k2" May 21 11:32:49.240886 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.240861 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f797c9ec-815c-4e1a-aa83-9173927d8593-service-ca-bundle\") pod \"insights-operator-78bdfb888c-49stn\" (UID: \"f797c9ec-815c-4e1a-aa83-9173927d8593\") " pod="openshift-insights/insights-operator-78bdfb888c-49stn" May 21 11:32:49.241200 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:32:49.240949 2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found May 21 11:32:49.241200 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.240972 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wpnjj\" (UniqueName: \"kubernetes.io/projected/4540e5a0-576b-4722-acfc-a2cec978f9e3-kube-api-access-wpnjj\") pod \"router-default-b7f67965d-272k2\" (UID: \"4540e5a0-576b-4722-acfc-a2cec978f9e3\") " pod="openshift-ingress/router-default-b7f67965d-272k2" May 21 11:32:49.241200 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:32:49.241057 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4540e5a0-576b-4722-acfc-a2cec978f9e3-metrics-certs podName:4540e5a0-576b-4722-acfc-a2cec978f9e3 nodeName:}" failed. No retries permitted until 2026-05-21 11:32:49.741006305 +0000 UTC m=+124.636289828 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4540e5a0-576b-4722-acfc-a2cec978f9e3-metrics-certs") pod "router-default-b7f67965d-272k2" (UID: "4540e5a0-576b-4722-acfc-a2cec978f9e3") : secret "router-metrics-certs-default" not found May 21 11:32:49.241200 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.241119 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f797c9ec-815c-4e1a-aa83-9173927d8593-trusted-ca-bundle\") pod \"insights-operator-78bdfb888c-49stn\" (UID: \"f797c9ec-815c-4e1a-aa83-9173927d8593\") " pod="openshift-insights/insights-operator-78bdfb888c-49stn" May 21 11:32:49.241200 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.241149 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2q74p\" (UniqueName: \"kubernetes.io/projected/89aba95e-fa54-4fba-bf2b-0e3569017e24-kube-api-access-2q74p\") pod \"cluster-samples-operator-697587599-vv4gd\" (UID: \"89aba95e-fa54-4fba-bf2b-0e3569017e24\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-vv4gd" May 21 11:32:49.241200 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.241177 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f797c9ec-815c-4e1a-aa83-9173927d8593-tmp\") pod \"insights-operator-78bdfb888c-49stn\" (UID: \"f797c9ec-815c-4e1a-aa83-9173927d8593\") " pod="openshift-insights/insights-operator-78bdfb888c-49stn" May 21 11:32:49.241200 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.241200 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/f797c9ec-815c-4e1a-aa83-9173927d8593-snapshots\") pod \"insights-operator-78bdfb888c-49stn\" (UID: \"f797c9ec-815c-4e1a-aa83-9173927d8593\") " pod="openshift-insights/insights-operator-78bdfb888c-49stn" May 21 11:32:49.241622 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.241247 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f797c9ec-815c-4e1a-aa83-9173927d8593-serving-cert\") pod \"insights-operator-78bdfb888c-49stn\" (UID: \"f797c9ec-815c-4e1a-aa83-9173927d8593\") " pod="openshift-insights/insights-operator-78bdfb888c-49stn" May 21 11:32:49.241622 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.241268 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dstn4\" (UniqueName: \"kubernetes.io/projected/f797c9ec-815c-4e1a-aa83-9173927d8593-kube-api-access-dstn4\") pod \"insights-operator-78bdfb888c-49stn\" (UID: \"f797c9ec-815c-4e1a-aa83-9173927d8593\") " pod="openshift-insights/insights-operator-78bdfb888c-49stn" May 21 11:32:49.241622 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.241299 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/4540e5a0-576b-4722-acfc-a2cec978f9e3-stats-auth\") pod \"router-default-b7f67965d-272k2\" (UID: \"4540e5a0-576b-4722-acfc-a2cec978f9e3\") " pod="openshift-ingress/router-default-b7f67965d-272k2" May 21 11:32:49.241622 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.241354 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4540e5a0-576b-4722-acfc-a2cec978f9e3-service-ca-bundle\") pod \"router-default-b7f67965d-272k2\" (UID: \"4540e5a0-576b-4722-acfc-a2cec978f9e3\") " pod="openshift-ingress/router-default-b7f67965d-272k2" May 21 11:32:49.241622 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.241385 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/89aba95e-fa54-4fba-bf2b-0e3569017e24-samples-operator-tls\") pod \"cluster-samples-operator-697587599-vv4gd\" (UID: \"89aba95e-fa54-4fba-bf2b-0e3569017e24\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-vv4gd" May 21 11:32:49.241622 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:32:49.241474 2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found May 21 11:32:49.241622 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.241484 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f797c9ec-815c-4e1a-aa83-9173927d8593-service-ca-bundle\") pod \"insights-operator-78bdfb888c-49stn\" (UID: \"f797c9ec-815c-4e1a-aa83-9173927d8593\") " pod="openshift-insights/insights-operator-78bdfb888c-49stn" May 21 11:32:49.241622 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.241506 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f797c9ec-815c-4e1a-aa83-9173927d8593-tmp\") pod \"insights-operator-78bdfb888c-49stn\" (UID: \"f797c9ec-815c-4e1a-aa83-9173927d8593\") " pod="openshift-insights/insights-operator-78bdfb888c-49stn" May 21 11:32:49.241622 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:32:49.241517 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89aba95e-fa54-4fba-bf2b-0e3569017e24-samples-operator-tls podName:89aba95e-fa54-4fba-bf2b-0e3569017e24 nodeName:}" failed. No retries permitted until 2026-05-21 11:32:49.741502648 +0000 UTC m=+124.636786171 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/89aba95e-fa54-4fba-bf2b-0e3569017e24-samples-operator-tls") pod "cluster-samples-operator-697587599-vv4gd" (UID: "89aba95e-fa54-4fba-bf2b-0e3569017e24") : secret "samples-operator-tls" not found May 21 11:32:49.241622 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:32:49.241676 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4540e5a0-576b-4722-acfc-a2cec978f9e3-service-ca-bundle podName:4540e5a0-576b-4722-acfc-a2cec978f9e3 nodeName:}" failed. No retries permitted until 2026-05-21 11:32:49.741664096 +0000 UTC m=+124.636947638 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/4540e5a0-576b-4722-acfc-a2cec978f9e3-service-ca-bundle") pod "router-default-b7f67965d-272k2" (UID: "4540e5a0-576b-4722-acfc-a2cec978f9e3") : configmap references non-existent config key: service-ca.crt May 21 11:32:49.242329 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.241846 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/f797c9ec-815c-4e1a-aa83-9173927d8593-snapshots\") pod \"insights-operator-78bdfb888c-49stn\" (UID: \"f797c9ec-815c-4e1a-aa83-9173927d8593\") " pod="openshift-insights/insights-operator-78bdfb888c-49stn" May 21 11:32:49.242329 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.242260 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f797c9ec-815c-4e1a-aa83-9173927d8593-trusted-ca-bundle\") pod \"insights-operator-78bdfb888c-49stn\" (UID: \"f797c9ec-815c-4e1a-aa83-9173927d8593\") " pod="openshift-insights/insights-operator-78bdfb888c-49stn" May 21 11:32:49.243716 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.243694 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f797c9ec-815c-4e1a-aa83-9173927d8593-serving-cert\") pod \"insights-operator-78bdfb888c-49stn\" (UID: \"f797c9ec-815c-4e1a-aa83-9173927d8593\") " pod="openshift-insights/insights-operator-78bdfb888c-49stn" May 21 11:32:49.243716 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.243711 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/4540e5a0-576b-4722-acfc-a2cec978f9e3-default-certificate\") pod \"router-default-b7f67965d-272k2\" (UID: \"4540e5a0-576b-4722-acfc-a2cec978f9e3\") " pod="openshift-ingress/router-default-b7f67965d-272k2" May 21 11:32:49.244150 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.244128 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/4540e5a0-576b-4722-acfc-a2cec978f9e3-stats-auth\") pod \"router-default-b7f67965d-272k2\" (UID: \"4540e5a0-576b-4722-acfc-a2cec978f9e3\") " pod="openshift-ingress/router-default-b7f67965d-272k2" May 21 11:32:49.252677 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.252651 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpnjj\" (UniqueName: \"kubernetes.io/projected/4540e5a0-576b-4722-acfc-a2cec978f9e3-kube-api-access-wpnjj\") pod \"router-default-b7f67965d-272k2\" (UID: \"4540e5a0-576b-4722-acfc-a2cec978f9e3\") " pod="openshift-ingress/router-default-b7f67965d-272k2" May 21 11:32:49.252829 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.252807 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2q74p\" (UniqueName: \"kubernetes.io/projected/89aba95e-fa54-4fba-bf2b-0e3569017e24-kube-api-access-2q74p\") pod \"cluster-samples-operator-697587599-vv4gd\" (UID: \"89aba95e-fa54-4fba-bf2b-0e3569017e24\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-vv4gd" May 21 11:32:49.252891 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.252817 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dstn4\" (UniqueName: \"kubernetes.io/projected/f797c9ec-815c-4e1a-aa83-9173927d8593-kube-api-access-dstn4\") pod \"insights-operator-78bdfb888c-49stn\" (UID: \"f797c9ec-815c-4e1a-aa83-9173927d8593\") " pod="openshift-insights/insights-operator-78bdfb888c-49stn" May 21 11:32:49.276769 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.276751 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-xdnz5" May 21 11:32:49.283059 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.283037 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-76dff48d59-6g9xk" May 21 11:32:49.390670 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.390638 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-78bdfb888c-49stn" May 21 11:32:49.407087 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.407042 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-xdnz5"] May 21 11:32:49.410068 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:32:49.410037 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda74a4dc1_ca52_40f4_9cb2_0c925a08ecfd.slice/crio-52e9daa1c2ed465f3edcb0b7fcc20a43e2c0efbe2ab917ff8759990b53171c00 WatchSource:0}: Error finding container 52e9daa1c2ed465f3edcb0b7fcc20a43e2c0efbe2ab917ff8759990b53171c00: Status 404 returned error can't find the container with id 52e9daa1c2ed465f3edcb0b7fcc20a43e2c0efbe2ab917ff8759990b53171c00 May 21 11:32:49.422854 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.422832 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-76dff48d59-6g9xk"] May 21 11:32:49.426590 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:32:49.426565 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe463222_9e99_47cb_bf6b_525913ca9e29.slice/crio-49588cd113d7c82bad1c356385449f09a17c40e51c2bd612bd772af4a27aac95 WatchSource:0}: Error finding container 49588cd113d7c82bad1c356385449f09a17c40e51c2bd612bd772af4a27aac95: Status 404 returned error can't find the container with id 49588cd113d7c82bad1c356385449f09a17c40e51c2bd612bd772af4a27aac95 May 21 11:32:49.514878 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.514806 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-78bdfb888c-49stn"] May 21 11:32:49.517903 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:32:49.517878 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf797c9ec_815c_4e1a_aa83_9173927d8593.slice/crio-6f3123d3c59d8286487f39fce9021ec4bc68fa3802df9cb5339fee2ade943c3a WatchSource:0}: Error finding container 6f3123d3c59d8286487f39fce9021ec4bc68fa3802df9cb5339fee2ade943c3a: Status 404 returned error can't find the container with id 6f3123d3c59d8286487f39fce9021ec4bc68fa3802df9cb5339fee2ade943c3a May 21 11:32:49.745640 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.745606 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4540e5a0-576b-4722-acfc-a2cec978f9e3-metrics-certs\") pod \"router-default-b7f67965d-272k2\" (UID: \"4540e5a0-576b-4722-acfc-a2cec978f9e3\") " pod="openshift-ingress/router-default-b7f67965d-272k2" May 21 11:32:49.745822 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.745698 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4540e5a0-576b-4722-acfc-a2cec978f9e3-service-ca-bundle\") pod \"router-default-b7f67965d-272k2\" (UID: \"4540e5a0-576b-4722-acfc-a2cec978f9e3\") " pod="openshift-ingress/router-default-b7f67965d-272k2" May 21 11:32:49.745822 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:49.745719 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/89aba95e-fa54-4fba-bf2b-0e3569017e24-samples-operator-tls\") pod \"cluster-samples-operator-697587599-vv4gd\" (UID: \"89aba95e-fa54-4fba-bf2b-0e3569017e24\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-vv4gd" May 21 11:32:49.745822 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:32:49.745754 2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found May 21 11:32:49.745994 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:32:49.745974 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4540e5a0-576b-4722-acfc-a2cec978f9e3-metrics-certs podName:4540e5a0-576b-4722-acfc-a2cec978f9e3 nodeName:}" failed. No retries permitted until 2026-05-21 11:32:50.745915748 +0000 UTC m=+125.641199284 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4540e5a0-576b-4722-acfc-a2cec978f9e3-metrics-certs") pod "router-default-b7f67965d-272k2" (UID: "4540e5a0-576b-4722-acfc-a2cec978f9e3") : secret "router-metrics-certs-default" not found May 21 11:32:49.748961 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:32:49.746142 2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found May 21 11:32:49.748961 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:32:49.746259 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4540e5a0-576b-4722-acfc-a2cec978f9e3-service-ca-bundle podName:4540e5a0-576b-4722-acfc-a2cec978f9e3 nodeName:}" failed. No retries permitted until 2026-05-21 11:32:50.746242527 +0000 UTC m=+125.641526053 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/4540e5a0-576b-4722-acfc-a2cec978f9e3-service-ca-bundle") pod "router-default-b7f67965d-272k2" (UID: "4540e5a0-576b-4722-acfc-a2cec978f9e3") : configmap references non-existent config key: service-ca.crt May 21 11:32:49.748961 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:32:49.746523 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89aba95e-fa54-4fba-bf2b-0e3569017e24-samples-operator-tls podName:89aba95e-fa54-4fba-bf2b-0e3569017e24 nodeName:}" failed. No retries permitted until 2026-05-21 11:32:50.746507652 +0000 UTC m=+125.641791193 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/89aba95e-fa54-4fba-bf2b-0e3569017e24-samples-operator-tls") pod "cluster-samples-operator-697587599-vv4gd" (UID: "89aba95e-fa54-4fba-bf2b-0e3569017e24") : secret "samples-operator-tls" not found May 21 11:32:50.096957 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:50.096914 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-76dff48d59-6g9xk" event={"ID":"be463222-9e99-47cb-bf6b-525913ca9e29","Type":"ContainerStarted","Data":"49588cd113d7c82bad1c356385449f09a17c40e51c2bd612bd772af4a27aac95"} May 21 11:32:50.098142 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:50.098109 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-78bdfb888c-49stn" event={"ID":"f797c9ec-815c-4e1a-aa83-9173927d8593","Type":"ContainerStarted","Data":"6f3123d3c59d8286487f39fce9021ec4bc68fa3802df9cb5339fee2ade943c3a"} May 21 11:32:50.099465 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:50.099429 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-xdnz5" event={"ID":"a74a4dc1-ca52-40f4-9cb2-0c925a08ecfd","Type":"ContainerStarted","Data":"52e9daa1c2ed465f3edcb0b7fcc20a43e2c0efbe2ab917ff8759990b53171c00"} May 21 11:32:50.754888 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:50.754830 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4540e5a0-576b-4722-acfc-a2cec978f9e3-service-ca-bundle\") pod \"router-default-b7f67965d-272k2\" (UID: \"4540e5a0-576b-4722-acfc-a2cec978f9e3\") " pod="openshift-ingress/router-default-b7f67965d-272k2" May 21 11:32:50.754888 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:50.754886 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/89aba95e-fa54-4fba-bf2b-0e3569017e24-samples-operator-tls\") pod \"cluster-samples-operator-697587599-vv4gd\" (UID: \"89aba95e-fa54-4fba-bf2b-0e3569017e24\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-vv4gd" May 21 11:32:50.755205 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:50.754922 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4540e5a0-576b-4722-acfc-a2cec978f9e3-metrics-certs\") pod \"router-default-b7f67965d-272k2\" (UID: \"4540e5a0-576b-4722-acfc-a2cec978f9e3\") " pod="openshift-ingress/router-default-b7f67965d-272k2" May 21 11:32:50.755205 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:32:50.755035 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4540e5a0-576b-4722-acfc-a2cec978f9e3-service-ca-bundle podName:4540e5a0-576b-4722-acfc-a2cec978f9e3 nodeName:}" failed. No retries permitted until 2026-05-21 11:32:52.755009492 +0000 UTC m=+127.650293036 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/4540e5a0-576b-4722-acfc-a2cec978f9e3-service-ca-bundle") pod "router-default-b7f67965d-272k2" (UID: "4540e5a0-576b-4722-acfc-a2cec978f9e3") : configmap references non-existent config key: service-ca.crt May 21 11:32:50.755205 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:32:50.755053 2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found May 21 11:32:50.755205 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:32:50.755136 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4540e5a0-576b-4722-acfc-a2cec978f9e3-metrics-certs podName:4540e5a0-576b-4722-acfc-a2cec978f9e3 nodeName:}" failed. No retries permitted until 2026-05-21 11:32:52.755119018 +0000 UTC m=+127.650402554 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4540e5a0-576b-4722-acfc-a2cec978f9e3-metrics-certs") pod "router-default-b7f67965d-272k2" (UID: "4540e5a0-576b-4722-acfc-a2cec978f9e3") : secret "router-metrics-certs-default" not found May 21 11:32:50.755205 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:32:50.755186 2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found May 21 11:32:50.755478 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:32:50.755220 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89aba95e-fa54-4fba-bf2b-0e3569017e24-samples-operator-tls podName:89aba95e-fa54-4fba-bf2b-0e3569017e24 nodeName:}" failed. No retries permitted until 2026-05-21 11:32:52.755209781 +0000 UTC m=+127.650493315 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/89aba95e-fa54-4fba-bf2b-0e3569017e24-samples-operator-tls") pod "cluster-samples-operator-697587599-vv4gd" (UID: "89aba95e-fa54-4fba-bf2b-0e3569017e24") : secret "samples-operator-tls" not found May 21 11:32:52.773434 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:52.773392 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4540e5a0-576b-4722-acfc-a2cec978f9e3-service-ca-bundle\") pod \"router-default-b7f67965d-272k2\" (UID: \"4540e5a0-576b-4722-acfc-a2cec978f9e3\") " pod="openshift-ingress/router-default-b7f67965d-272k2" May 21 11:32:52.773434 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:52.773439 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/89aba95e-fa54-4fba-bf2b-0e3569017e24-samples-operator-tls\") pod \"cluster-samples-operator-697587599-vv4gd\" (UID: \"89aba95e-fa54-4fba-bf2b-0e3569017e24\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-vv4gd" May 21 11:32:52.773991 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:52.773459 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4540e5a0-576b-4722-acfc-a2cec978f9e3-metrics-certs\") pod \"router-default-b7f67965d-272k2\" (UID: \"4540e5a0-576b-4722-acfc-a2cec978f9e3\") " pod="openshift-ingress/router-default-b7f67965d-272k2" May 21 11:32:52.773991 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:32:52.773577 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4540e5a0-576b-4722-acfc-a2cec978f9e3-service-ca-bundle podName:4540e5a0-576b-4722-acfc-a2cec978f9e3 nodeName:}" failed. No retries permitted until 2026-05-21 11:32:56.773560078 +0000 UTC m=+131.668843601 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/4540e5a0-576b-4722-acfc-a2cec978f9e3-service-ca-bundle") pod "router-default-b7f67965d-272k2" (UID: "4540e5a0-576b-4722-acfc-a2cec978f9e3") : configmap references non-existent config key: service-ca.crt May 21 11:32:52.773991 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:32:52.773578 2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found May 21 11:32:52.773991 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:32:52.773622 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89aba95e-fa54-4fba-bf2b-0e3569017e24-samples-operator-tls podName:89aba95e-fa54-4fba-bf2b-0e3569017e24 nodeName:}" failed. No retries permitted until 2026-05-21 11:32:56.773615497 +0000 UTC m=+131.668899020 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/89aba95e-fa54-4fba-bf2b-0e3569017e24-samples-operator-tls") pod "cluster-samples-operator-697587599-vv4gd" (UID: "89aba95e-fa54-4fba-bf2b-0e3569017e24") : secret "samples-operator-tls" not found May 21 11:32:52.773991 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:32:52.773582 2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found May 21 11:32:52.773991 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:32:52.773704 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4540e5a0-576b-4722-acfc-a2cec978f9e3-metrics-certs podName:4540e5a0-576b-4722-acfc-a2cec978f9e3 nodeName:}" failed. No retries permitted until 2026-05-21 11:32:56.773684142 +0000 UTC m=+131.668967678 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4540e5a0-576b-4722-acfc-a2cec978f9e3-metrics-certs") pod "router-default-b7f67965d-272k2" (UID: "4540e5a0-576b-4722-acfc-a2cec978f9e3") : secret "router-metrics-certs-default" not found May 21 11:32:53.107267 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:53.107228 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-xdnz5" event={"ID":"a74a4dc1-ca52-40f4-9cb2-0c925a08ecfd","Type":"ContainerStarted","Data":"367fe8458136048878f86af42ecf2ad42f4311c74793ef8c785cb138661156b1"} May 21 11:32:53.108708 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:53.108683 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-6g9xk_be463222-9e99-47cb-bf6b-525913ca9e29/console-operator/0.log" May 21 11:32:53.108839 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:53.108729 2564 generic.go:358] "Generic (PLEG): container finished" podID="be463222-9e99-47cb-bf6b-525913ca9e29" containerID="f058c5d30fd2d1129ff8be60c491eb672b565cb7475aedda5eab90a3431fb112" exitCode=255 May 21 11:32:53.108839 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:53.108799 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-76dff48d59-6g9xk" event={"ID":"be463222-9e99-47cb-bf6b-525913ca9e29","Type":"ContainerDied","Data":"f058c5d30fd2d1129ff8be60c491eb672b565cb7475aedda5eab90a3431fb112"} May 21 11:32:53.109037 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:53.109021 2564 scope.go:117] "RemoveContainer" containerID="f058c5d30fd2d1129ff8be60c491eb672b565cb7475aedda5eab90a3431fb112" May 21 11:32:53.110278 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:53.110253 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-78bdfb888c-49stn" event={"ID":"f797c9ec-815c-4e1a-aa83-9173927d8593","Type":"ContainerStarted","Data":"f47f02fae257a346d5a29829a184f682848eb8c5c03db5606ccffeeb24808489"} May 21 11:32:53.135919 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:53.135879 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-xdnz5" podStartSLOduration=2.344829173 podStartE2EDuration="5.135866356s" podCreationTimestamp="2026-05-21 11:32:48 +0000 UTC" firstStartedPulling="2026-05-21 11:32:49.412597704 +0000 UTC m=+124.307881244" lastFinishedPulling="2026-05-21 11:32:52.203634886 +0000 UTC m=+127.098918427" observedRunningTime="2026-05-21 11:32:53.134802145 +0000 UTC m=+128.030085690" watchObservedRunningTime="2026-05-21 11:32:53.135866356 +0000 UTC m=+128.031149900" May 21 11:32:53.164521 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:53.164470 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-operator-78bdfb888c-49stn" podStartSLOduration=1.479564568 podStartE2EDuration="4.164452613s" podCreationTimestamp="2026-05-21 11:32:49 +0000 UTC" firstStartedPulling="2026-05-21 11:32:49.519728182 +0000 UTC m=+124.415011705" lastFinishedPulling="2026-05-21 11:32:52.204616227 +0000 UTC m=+127.099899750" observedRunningTime="2026-05-21 11:32:53.163573595 +0000 UTC m=+128.058857140" watchObservedRunningTime="2026-05-21 11:32:53.164452613 +0000 UTC m=+128.059736159" May 21 11:32:54.114290 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:54.114262 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-6g9xk_be463222-9e99-47cb-bf6b-525913ca9e29/console-operator/1.log" May 21 11:32:54.114700 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:54.114605 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-6g9xk_be463222-9e99-47cb-bf6b-525913ca9e29/console-operator/0.log" May 21 11:32:54.114700 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:54.114636 2564 generic.go:358] "Generic (PLEG): container finished" podID="be463222-9e99-47cb-bf6b-525913ca9e29" containerID="1c53ed40b4a8ab52596f3975b0c822631c7ec3ca99dc6acf0fa336ff2e20154a" exitCode=255 May 21 11:32:54.114817 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:54.114734 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-76dff48d59-6g9xk" event={"ID":"be463222-9e99-47cb-bf6b-525913ca9e29","Type":"ContainerDied","Data":"1c53ed40b4a8ab52596f3975b0c822631c7ec3ca99dc6acf0fa336ff2e20154a"} May 21 11:32:54.114817 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:54.114776 2564 scope.go:117] "RemoveContainer" containerID="f058c5d30fd2d1129ff8be60c491eb672b565cb7475aedda5eab90a3431fb112" May 21 11:32:54.115027 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:54.114998 2564 scope.go:117] "RemoveContainer" containerID="1c53ed40b4a8ab52596f3975b0c822631c7ec3ca99dc6acf0fa336ff2e20154a" May 21 11:32:54.115238 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:32:54.115218 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-76dff48d59-6g9xk_openshift-console-operator(be463222-9e99-47cb-bf6b-525913ca9e29)\"" pod="openshift-console-operator/console-operator-76dff48d59-6g9xk" podUID="be463222-9e99-47cb-bf6b-525913ca9e29" May 21 11:32:54.489260 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:54.489167 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/414e12fe-b9ac-4bbf-8470-ad442215f5cb-metrics-certs\") pod \"network-metrics-daemon-xf5gc\" (UID: \"414e12fe-b9ac-4bbf-8470-ad442215f5cb\") " pod="openshift-multus/network-metrics-daemon-xf5gc" May 21 11:32:54.489408 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:32:54.489321 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found May 21 11:32:54.489408 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:32:54.489401 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/414e12fe-b9ac-4bbf-8470-ad442215f5cb-metrics-certs podName:414e12fe-b9ac-4bbf-8470-ad442215f5cb nodeName:}" failed. No retries permitted until 2026-05-21 11:34:56.489380537 +0000 UTC m=+251.384664072 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/414e12fe-b9ac-4bbf-8470-ad442215f5cb-metrics-certs") pod "network-metrics-daemon-xf5gc" (UID: "414e12fe-b9ac-4bbf-8470-ad442215f5cb") : secret "metrics-daemon-secret" not found May 21 11:32:55.117738 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:55.117707 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-6g9xk_be463222-9e99-47cb-bf6b-525913ca9e29/console-operator/1.log" May 21 11:32:55.118135 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:55.118113 2564 scope.go:117] "RemoveContainer" containerID="1c53ed40b4a8ab52596f3975b0c822631c7ec3ca99dc6acf0fa336ff2e20154a" May 21 11:32:55.118325 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:32:55.118307 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-76dff48d59-6g9xk_openshift-console-operator(be463222-9e99-47cb-bf6b-525913ca9e29)\"" pod="openshift-console-operator/console-operator-76dff48d59-6g9xk" podUID="be463222-9e99-47cb-bf6b-525913ca9e29" May 21 11:32:55.369250 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:55.369172 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-67d667c7d-gw6tc"] May 21 11:32:55.371626 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:55.371608 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-67d667c7d-gw6tc" May 21 11:32:55.375457 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:55.375435 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" May 21 11:32:55.375564 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:55.375503 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-6jn99\"" May 21 11:32:55.375732 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:55.375719 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" May 21 11:32:55.376822 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:55.376803 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" May 21 11:32:55.376930 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:55.376848 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" May 21 11:32:55.384522 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:55.384500 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-67d667c7d-gw6tc"] May 21 11:32:55.496295 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:55.496255 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c2fa73f9-25f2-4102-981e-8fea3e3f0414-signing-key\") pod \"service-ca-67d667c7d-gw6tc\" (UID: \"c2fa73f9-25f2-4102-981e-8fea3e3f0414\") " pod="openshift-service-ca/service-ca-67d667c7d-gw6tc" May 21 11:32:55.496476 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:55.496300 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q46b5\" (UniqueName: \"kubernetes.io/projected/c2fa73f9-25f2-4102-981e-8fea3e3f0414-kube-api-access-q46b5\") pod \"service-ca-67d667c7d-gw6tc\" (UID: \"c2fa73f9-25f2-4102-981e-8fea3e3f0414\") " pod="openshift-service-ca/service-ca-67d667c7d-gw6tc" May 21 11:32:55.496476 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:55.496438 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c2fa73f9-25f2-4102-981e-8fea3e3f0414-signing-cabundle\") pod \"service-ca-67d667c7d-gw6tc\" (UID: \"c2fa73f9-25f2-4102-981e-8fea3e3f0414\") " pod="openshift-service-ca/service-ca-67d667c7d-gw6tc" May 21 11:32:55.596977 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:55.596936 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c2fa73f9-25f2-4102-981e-8fea3e3f0414-signing-cabundle\") pod \"service-ca-67d667c7d-gw6tc\" (UID: \"c2fa73f9-25f2-4102-981e-8fea3e3f0414\") " pod="openshift-service-ca/service-ca-67d667c7d-gw6tc" May 21 11:32:55.597172 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:55.597031 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c2fa73f9-25f2-4102-981e-8fea3e3f0414-signing-key\") pod \"service-ca-67d667c7d-gw6tc\" (UID: \"c2fa73f9-25f2-4102-981e-8fea3e3f0414\") " pod="openshift-service-ca/service-ca-67d667c7d-gw6tc" May 21 11:32:55.597172 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:55.597057 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-q46b5\" (UniqueName: \"kubernetes.io/projected/c2fa73f9-25f2-4102-981e-8fea3e3f0414-kube-api-access-q46b5\") pod \"service-ca-67d667c7d-gw6tc\" (UID: \"c2fa73f9-25f2-4102-981e-8fea3e3f0414\") " pod="openshift-service-ca/service-ca-67d667c7d-gw6tc" May 21 11:32:55.597688 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:55.597671 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c2fa73f9-25f2-4102-981e-8fea3e3f0414-signing-cabundle\") pod \"service-ca-67d667c7d-gw6tc\" (UID: \"c2fa73f9-25f2-4102-981e-8fea3e3f0414\") " pod="openshift-service-ca/service-ca-67d667c7d-gw6tc" May 21 11:32:55.599611 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:55.599589 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c2fa73f9-25f2-4102-981e-8fea3e3f0414-signing-key\") pod \"service-ca-67d667c7d-gw6tc\" (UID: \"c2fa73f9-25f2-4102-981e-8fea3e3f0414\") " pod="openshift-service-ca/service-ca-67d667c7d-gw6tc" May 21 11:32:55.600386 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:55.600370 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lkfnm_71722934-df74-4aa7-91e7-9f388d432991/dns-node-resolver/0.log" May 21 11:32:55.606521 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:55.606501 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q46b5\" (UniqueName: \"kubernetes.io/projected/c2fa73f9-25f2-4102-981e-8fea3e3f0414-kube-api-access-q46b5\") pod \"service-ca-67d667c7d-gw6tc\" (UID: \"c2fa73f9-25f2-4102-981e-8fea3e3f0414\") " pod="openshift-service-ca/service-ca-67d667c7d-gw6tc" May 21 11:32:55.680304 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:55.680215 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-67d667c7d-gw6tc" May 21 11:32:55.803880 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:55.803850 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-67d667c7d-gw6tc"] May 21 11:32:55.807545 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:32:55.807518 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2fa73f9_25f2_4102_981e_8fea3e3f0414.slice/crio-e3a28ee9f27f9e43d425c16d714c097f374d4fe8ac411c14ebe88aeee8385f20 WatchSource:0}: Error finding container e3a28ee9f27f9e43d425c16d714c097f374d4fe8ac411c14ebe88aeee8385f20: Status 404 returned error can't find the container with id e3a28ee9f27f9e43d425c16d714c097f374d4fe8ac411c14ebe88aeee8385f20 May 21 11:32:56.120561 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:56.120529 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-67d667c7d-gw6tc" event={"ID":"c2fa73f9-25f2-4102-981e-8fea3e3f0414","Type":"ContainerStarted","Data":"e3a28ee9f27f9e43d425c16d714c097f374d4fe8ac411c14ebe88aeee8385f20"} May 21 11:32:56.400856 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:56.400780 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-29k74_0fd57831-cff3-4f96-8695-cb4d02bc95a9/node-ca/0.log" May 21 11:32:56.806046 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:56.806008 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4540e5a0-576b-4722-acfc-a2cec978f9e3-service-ca-bundle\") pod \"router-default-b7f67965d-272k2\" (UID: \"4540e5a0-576b-4722-acfc-a2cec978f9e3\") " pod="openshift-ingress/router-default-b7f67965d-272k2" May 21 11:32:56.806256 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:56.806071 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/89aba95e-fa54-4fba-bf2b-0e3569017e24-samples-operator-tls\") pod \"cluster-samples-operator-697587599-vv4gd\" (UID: \"89aba95e-fa54-4fba-bf2b-0e3569017e24\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-vv4gd" May 21 11:32:56.806256 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:56.806127 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4540e5a0-576b-4722-acfc-a2cec978f9e3-metrics-certs\") pod \"router-default-b7f67965d-272k2\" (UID: \"4540e5a0-576b-4722-acfc-a2cec978f9e3\") " pod="openshift-ingress/router-default-b7f67965d-272k2" May 21 11:32:56.806491 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:32:56.806322 2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found May 21 11:32:56.806491 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:32:56.806400 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4540e5a0-576b-4722-acfc-a2cec978f9e3-metrics-certs podName:4540e5a0-576b-4722-acfc-a2cec978f9e3 nodeName:}" failed. No retries permitted until 2026-05-21 11:33:04.806377672 +0000 UTC m=+139.701661208 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4540e5a0-576b-4722-acfc-a2cec978f9e3-metrics-certs") pod "router-default-b7f67965d-272k2" (UID: "4540e5a0-576b-4722-acfc-a2cec978f9e3") : secret "router-metrics-certs-default" not found May 21 11:32:56.806491 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:32:56.806407 2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found May 21 11:32:56.806491 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:32:56.806420 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4540e5a0-576b-4722-acfc-a2cec978f9e3-service-ca-bundle podName:4540e5a0-576b-4722-acfc-a2cec978f9e3 nodeName:}" failed. No retries permitted until 2026-05-21 11:33:04.806409968 +0000 UTC m=+139.701693493 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/4540e5a0-576b-4722-acfc-a2cec978f9e3-service-ca-bundle") pod "router-default-b7f67965d-272k2" (UID: "4540e5a0-576b-4722-acfc-a2cec978f9e3") : configmap references non-existent config key: service-ca.crt May 21 11:32:56.806491 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:32:56.806474 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89aba95e-fa54-4fba-bf2b-0e3569017e24-samples-operator-tls podName:89aba95e-fa54-4fba-bf2b-0e3569017e24 nodeName:}" failed. No retries permitted until 2026-05-21 11:33:04.806456537 +0000 UTC m=+139.701740066 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/89aba95e-fa54-4fba-bf2b-0e3569017e24-samples-operator-tls") pod "cluster-samples-operator-697587599-vv4gd" (UID: "89aba95e-fa54-4fba-bf2b-0e3569017e24") : secret "samples-operator-tls" not found May 21 11:32:58.125759 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:58.125713 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-67d667c7d-gw6tc" event={"ID":"c2fa73f9-25f2-4102-981e-8fea3e3f0414","Type":"ContainerStarted","Data":"100fa3570d3ee224980755ee49a040b032251ccf5fa730b2728501e4a858997a"} May 21 11:32:58.148900 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:58.148855 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-service-ca/service-ca-67d667c7d-gw6tc" podStartSLOduration=1.4191921220000001 podStartE2EDuration="3.148841563s" podCreationTimestamp="2026-05-21 11:32:55 +0000 UTC" firstStartedPulling="2026-05-21 11:32:55.80940433 +0000 UTC m=+130.704687855" lastFinishedPulling="2026-05-21 11:32:57.53905376 +0000 UTC m=+132.434337296" observedRunningTime="2026-05-21 11:32:58.148266154 +0000 UTC m=+133.043549698" watchObservedRunningTime="2026-05-21 11:32:58.148841563 +0000 UTC m=+133.044125108" May 21 11:32:59.283394 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:59.283361 2564 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-76dff48d59-6g9xk" May 21 11:32:59.283394 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:59.283409 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-76dff48d59-6g9xk" May 21 11:32:59.283821 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:32:59.283781 2564 scope.go:117] "RemoveContainer" containerID="1c53ed40b4a8ab52596f3975b0c822631c7ec3ca99dc6acf0fa336ff2e20154a" May 21 11:32:59.283956 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:32:59.283939 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-76dff48d59-6g9xk_openshift-console-operator(be463222-9e99-47cb-bf6b-525913ca9e29)\"" pod="openshift-console-operator/console-operator-76dff48d59-6g9xk" podUID="be463222-9e99-47cb-bf6b-525913ca9e29" May 21 11:33:04.872391 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:04.872348 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4540e5a0-576b-4722-acfc-a2cec978f9e3-metrics-certs\") pod \"router-default-b7f67965d-272k2\" (UID: \"4540e5a0-576b-4722-acfc-a2cec978f9e3\") " pod="openshift-ingress/router-default-b7f67965d-272k2" May 21 11:33:04.872831 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:04.872566 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4540e5a0-576b-4722-acfc-a2cec978f9e3-service-ca-bundle\") pod \"router-default-b7f67965d-272k2\" (UID: \"4540e5a0-576b-4722-acfc-a2cec978f9e3\") " pod="openshift-ingress/router-default-b7f67965d-272k2" May 21 11:33:04.872831 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:04.872590 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/89aba95e-fa54-4fba-bf2b-0e3569017e24-samples-operator-tls\") pod \"cluster-samples-operator-697587599-vv4gd\" (UID: \"89aba95e-fa54-4fba-bf2b-0e3569017e24\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-vv4gd" May 21 11:33:04.872831 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:33:04.872771 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4540e5a0-576b-4722-acfc-a2cec978f9e3-service-ca-bundle podName:4540e5a0-576b-4722-acfc-a2cec978f9e3 nodeName:}" failed. No retries permitted until 2026-05-21 11:33:20.872746401 +0000 UTC m=+155.768029941 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/4540e5a0-576b-4722-acfc-a2cec978f9e3-service-ca-bundle") pod "router-default-b7f67965d-272k2" (UID: "4540e5a0-576b-4722-acfc-a2cec978f9e3") : configmap references non-existent config key: service-ca.crt May 21 11:33:04.874877 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:04.874851 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4540e5a0-576b-4722-acfc-a2cec978f9e3-metrics-certs\") pod \"router-default-b7f67965d-272k2\" (UID: \"4540e5a0-576b-4722-acfc-a2cec978f9e3\") " pod="openshift-ingress/router-default-b7f67965d-272k2" May 21 11:33:04.874998 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:04.874915 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/89aba95e-fa54-4fba-bf2b-0e3569017e24-samples-operator-tls\") pod \"cluster-samples-operator-697587599-vv4gd\" (UID: \"89aba95e-fa54-4fba-bf2b-0e3569017e24\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-vv4gd" May 21 11:33:04.983198 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:04.983148 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-vv4gd" May 21 11:33:05.117795 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:05.117761 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-697587599-vv4gd"] May 21 11:33:06.150119 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:06.150060 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-vv4gd" event={"ID":"89aba95e-fa54-4fba-bf2b-0e3569017e24","Type":"ContainerStarted","Data":"f47400b284b62a095a1b6f04cd6a7ca5f67fee744fdd01c11c183f6464760800"} May 21 11:33:07.154269 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:07.154234 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-vv4gd" event={"ID":"89aba95e-fa54-4fba-bf2b-0e3569017e24","Type":"ContainerStarted","Data":"ce42bbbff5f56122d923e61ce3bc1854b4d5edc921fd2bf687346c5ff27515b9"} May 21 11:33:07.154269 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:07.154271 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-vv4gd" event={"ID":"89aba95e-fa54-4fba-bf2b-0e3569017e24","Type":"ContainerStarted","Data":"a19318075387752de06366a8985803903a68c6666051055443695c4d04ab2c20"} May 21 11:33:07.174504 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:07.174460 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-vv4gd" podStartSLOduration=16.592369242 podStartE2EDuration="18.174447531s" podCreationTimestamp="2026-05-21 11:32:49 +0000 UTC" firstStartedPulling="2026-05-21 11:33:05.157360949 +0000 UTC m=+140.052644475" lastFinishedPulling="2026-05-21 11:33:06.739439227 +0000 UTC m=+141.634722764" observedRunningTime="2026-05-21 11:33:07.172644187 +0000 UTC m=+142.067927732" watchObservedRunningTime="2026-05-21 11:33:07.174447531 +0000 UTC m=+142.069731076" May 21 11:33:11.680242 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:11.680206 2564 scope.go:117] "RemoveContainer" containerID="1c53ed40b4a8ab52596f3975b0c822631c7ec3ca99dc6acf0fa336ff2e20154a" May 21 11:33:12.167646 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:12.167621 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-6g9xk_be463222-9e99-47cb-bf6b-525913ca9e29/console-operator/2.log" May 21 11:33:12.168019 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:12.168002 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-6g9xk_be463222-9e99-47cb-bf6b-525913ca9e29/console-operator/1.log" May 21 11:33:12.168145 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:12.168041 2564 generic.go:358] "Generic (PLEG): container finished" podID="be463222-9e99-47cb-bf6b-525913ca9e29" containerID="f06f3aafcc7dec3728eaad07da827eb505ef43d0d3542e47d5fd014562d0d7da" exitCode=255 May 21 11:33:12.168145 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:12.168100 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-76dff48d59-6g9xk" event={"ID":"be463222-9e99-47cb-bf6b-525913ca9e29","Type":"ContainerDied","Data":"f06f3aafcc7dec3728eaad07da827eb505ef43d0d3542e47d5fd014562d0d7da"} May 21 11:33:12.168145 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:12.168142 2564 scope.go:117] "RemoveContainer" containerID="1c53ed40b4a8ab52596f3975b0c822631c7ec3ca99dc6acf0fa336ff2e20154a" May 21 11:33:12.168512 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:12.168498 2564 scope.go:117] "RemoveContainer" containerID="f06f3aafcc7dec3728eaad07da827eb505ef43d0d3542e47d5fd014562d0d7da" May 21 11:33:12.168738 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:33:12.168706 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-76dff48d59-6g9xk_openshift-console-operator(be463222-9e99-47cb-bf6b-525913ca9e29)\"" pod="openshift-console-operator/console-operator-76dff48d59-6g9xk" podUID="be463222-9e99-47cb-bf6b-525913ca9e29" May 21 11:33:13.172227 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:13.172193 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-6g9xk_be463222-9e99-47cb-bf6b-525913ca9e29/console-operator/2.log" May 21 11:33:15.736606 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:15.736577 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-jq45s"] May 21 11:33:15.739355 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:15.739338 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jq45s" May 21 11:33:15.745101 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:15.745038 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6sbs9\"" May 21 11:33:15.745101 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:15.745064 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" May 21 11:33:15.745101 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:15.745068 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" May 21 11:33:15.760552 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:15.760516 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jq45s"] May 21 11:33:15.856115 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:15.856061 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/684e8f1e-9fe0-43a7-8c45-6ae9fde9e165-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jq45s\" (UID: \"684e8f1e-9fe0-43a7-8c45-6ae9fde9e165\") " pod="openshift-insights/insights-runtime-extractor-jq45s" May 21 11:33:15.856115 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:15.856115 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/684e8f1e-9fe0-43a7-8c45-6ae9fde9e165-crio-socket\") pod \"insights-runtime-extractor-jq45s\" (UID: \"684e8f1e-9fe0-43a7-8c45-6ae9fde9e165\") " pod="openshift-insights/insights-runtime-extractor-jq45s" May 21 11:33:15.856328 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:15.856239 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mfvn\" (UniqueName: \"kubernetes.io/projected/684e8f1e-9fe0-43a7-8c45-6ae9fde9e165-kube-api-access-9mfvn\") pod \"insights-runtime-extractor-jq45s\" (UID: \"684e8f1e-9fe0-43a7-8c45-6ae9fde9e165\") " pod="openshift-insights/insights-runtime-extractor-jq45s" May 21 11:33:15.856328 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:15.856277 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/684e8f1e-9fe0-43a7-8c45-6ae9fde9e165-data-volume\") pod \"insights-runtime-extractor-jq45s\" (UID: \"684e8f1e-9fe0-43a7-8c45-6ae9fde9e165\") " pod="openshift-insights/insights-runtime-extractor-jq45s" May 21 11:33:15.856328 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:15.856309 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/684e8f1e-9fe0-43a7-8c45-6ae9fde9e165-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jq45s\" (UID: \"684e8f1e-9fe0-43a7-8c45-6ae9fde9e165\") " pod="openshift-insights/insights-runtime-extractor-jq45s" May 21 11:33:15.957660 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:15.957628 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9mfvn\" (UniqueName: \"kubernetes.io/projected/684e8f1e-9fe0-43a7-8c45-6ae9fde9e165-kube-api-access-9mfvn\") pod \"insights-runtime-extractor-jq45s\" (UID: \"684e8f1e-9fe0-43a7-8c45-6ae9fde9e165\") " pod="openshift-insights/insights-runtime-extractor-jq45s" May 21 11:33:15.957660 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:15.957667 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/684e8f1e-9fe0-43a7-8c45-6ae9fde9e165-data-volume\") pod \"insights-runtime-extractor-jq45s\" (UID: \"684e8f1e-9fe0-43a7-8c45-6ae9fde9e165\") " pod="openshift-insights/insights-runtime-extractor-jq45s" May 21 11:33:15.957871 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:15.957707 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/684e8f1e-9fe0-43a7-8c45-6ae9fde9e165-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jq45s\" (UID: \"684e8f1e-9fe0-43a7-8c45-6ae9fde9e165\") " pod="openshift-insights/insights-runtime-extractor-jq45s" May 21 11:33:15.957871 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:15.957769 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/684e8f1e-9fe0-43a7-8c45-6ae9fde9e165-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jq45s\" (UID: \"684e8f1e-9fe0-43a7-8c45-6ae9fde9e165\") " pod="openshift-insights/insights-runtime-extractor-jq45s" May 21 11:33:15.957871 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:15.957793 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/684e8f1e-9fe0-43a7-8c45-6ae9fde9e165-crio-socket\") pod \"insights-runtime-extractor-jq45s\" (UID: \"684e8f1e-9fe0-43a7-8c45-6ae9fde9e165\") " pod="openshift-insights/insights-runtime-extractor-jq45s" May 21 11:33:15.957871 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:15.957870 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/684e8f1e-9fe0-43a7-8c45-6ae9fde9e165-crio-socket\") pod \"insights-runtime-extractor-jq45s\" (UID: \"684e8f1e-9fe0-43a7-8c45-6ae9fde9e165\") " pod="openshift-insights/insights-runtime-extractor-jq45s" May 21 11:33:15.958070 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:15.958001 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/684e8f1e-9fe0-43a7-8c45-6ae9fde9e165-data-volume\") pod \"insights-runtime-extractor-jq45s\" (UID: \"684e8f1e-9fe0-43a7-8c45-6ae9fde9e165\") " pod="openshift-insights/insights-runtime-extractor-jq45s" May 21 11:33:15.958349 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:15.958329 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/684e8f1e-9fe0-43a7-8c45-6ae9fde9e165-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jq45s\" (UID: \"684e8f1e-9fe0-43a7-8c45-6ae9fde9e165\") " pod="openshift-insights/insights-runtime-extractor-jq45s" May 21 11:33:15.960320 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:15.960288 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/684e8f1e-9fe0-43a7-8c45-6ae9fde9e165-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jq45s\" (UID: \"684e8f1e-9fe0-43a7-8c45-6ae9fde9e165\") " pod="openshift-insights/insights-runtime-extractor-jq45s" May 21 11:33:15.975367 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:15.975340 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mfvn\" (UniqueName: \"kubernetes.io/projected/684e8f1e-9fe0-43a7-8c45-6ae9fde9e165-kube-api-access-9mfvn\") pod \"insights-runtime-extractor-jq45s\" (UID: \"684e8f1e-9fe0-43a7-8c45-6ae9fde9e165\") " pod="openshift-insights/insights-runtime-extractor-jq45s" May 21 11:33:16.047535 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:16.047513 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jq45s" May 21 11:33:16.170739 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:16.170706 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jq45s"] May 21 11:33:16.173588 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:33:16.173565 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod684e8f1e_9fe0_43a7_8c45_6ae9fde9e165.slice/crio-7aa3a72384d95ab80e35147002119dbcd9cc86e67d3ca1f938339d00adc85efe WatchSource:0}: Error finding container 7aa3a72384d95ab80e35147002119dbcd9cc86e67d3ca1f938339d00adc85efe: Status 404 returned error can't find the container with id 7aa3a72384d95ab80e35147002119dbcd9cc86e67d3ca1f938339d00adc85efe May 21 11:33:16.180020 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:16.179991 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jq45s" event={"ID":"684e8f1e-9fe0-43a7-8c45-6ae9fde9e165","Type":"ContainerStarted","Data":"7aa3a72384d95ab80e35147002119dbcd9cc86e67d3ca1f938339d00adc85efe"} May 21 11:33:17.183549 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:17.183466 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jq45s" event={"ID":"684e8f1e-9fe0-43a7-8c45-6ae9fde9e165","Type":"ContainerStarted","Data":"3b15b23777e11c6883036f71a066e80602642c9cd580e12749645f085d267dfa"} May 21 11:33:17.183549 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:17.183509 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jq45s" event={"ID":"684e8f1e-9fe0-43a7-8c45-6ae9fde9e165","Type":"ContainerStarted","Data":"aeb43c5cf7a37c4b1a48173be8d1057b76d372a9d99fb7a541ec2d64f6817bf0"} May 21 11:33:19.190332 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:19.190293 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jq45s" event={"ID":"684e8f1e-9fe0-43a7-8c45-6ae9fde9e165","Type":"ContainerStarted","Data":"0abf5427926b61f8b5523dc76b61653732732a39c581675627ae45bad60168b6"} May 21 11:33:19.213187 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:19.213139 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-jq45s" podStartSLOduration=2.175514034 podStartE2EDuration="4.213125955s" podCreationTimestamp="2026-05-21 11:33:15 +0000 UTC" firstStartedPulling="2026-05-21 11:33:16.221043678 +0000 UTC m=+151.116327200" lastFinishedPulling="2026-05-21 11:33:18.258655596 +0000 UTC m=+153.153939121" observedRunningTime="2026-05-21 11:33:19.212193657 +0000 UTC m=+154.107477204" watchObservedRunningTime="2026-05-21 11:33:19.213125955 +0000 UTC m=+154.108409500" May 21 11:33:19.283383 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:19.283344 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-76dff48d59-6g9xk" May 21 11:33:19.283505 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:19.283406 2564 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-76dff48d59-6g9xk" May 21 11:33:19.283787 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:19.283770 2564 scope.go:117] "RemoveContainer" containerID="f06f3aafcc7dec3728eaad07da827eb505ef43d0d3542e47d5fd014562d0d7da" May 21 11:33:19.283974 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:33:19.283956 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-76dff48d59-6g9xk_openshift-console-operator(be463222-9e99-47cb-bf6b-525913ca9e29)\"" pod="openshift-console-operator/console-operator-76dff48d59-6g9xk" podUID="be463222-9e99-47cb-bf6b-525913ca9e29" May 21 11:33:20.897658 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:20.897622 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4540e5a0-576b-4722-acfc-a2cec978f9e3-service-ca-bundle\") pod \"router-default-b7f67965d-272k2\" (UID: \"4540e5a0-576b-4722-acfc-a2cec978f9e3\") " pod="openshift-ingress/router-default-b7f67965d-272k2" May 21 11:33:20.898215 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:20.898195 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4540e5a0-576b-4722-acfc-a2cec978f9e3-service-ca-bundle\") pod \"router-default-b7f67965d-272k2\" (UID: \"4540e5a0-576b-4722-acfc-a2cec978f9e3\") " pod="openshift-ingress/router-default-b7f67965d-272k2" May 21 11:33:21.195967 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:21.195888 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-b7f67965d-272k2" May 21 11:33:21.320314 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:21.320284 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-b7f67965d-272k2"] May 21 11:33:21.324042 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:33:21.324013 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4540e5a0_576b_4722_acfc_a2cec978f9e3.slice/crio-e644e6b6ddf3da36cf3e4e5efa9770b3c78898987c71ec1de7f97f14aa1f2f88 WatchSource:0}: Error finding container e644e6b6ddf3da36cf3e4e5efa9770b3c78898987c71ec1de7f97f14aa1f2f88: Status 404 returned error can't find the container with id e644e6b6ddf3da36cf3e4e5efa9770b3c78898987c71ec1de7f97f14aa1f2f88 May 21 11:33:22.020710 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:33:22.020656 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-s79qs" podUID="b14cbac4-0b39-404a-a413-a5ccf5fb6472" May 21 11:33:22.025912 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:33:22.025885 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-2pfk6" podUID="2190640c-a631-4c98-8d97-ade39bb1567a" May 21 11:33:22.198018 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:22.197986 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2pfk6" May 21 11:33:22.198018 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:22.197994 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-b7f67965d-272k2" event={"ID":"4540e5a0-576b-4722-acfc-a2cec978f9e3","Type":"ContainerStarted","Data":"84c4275524dbca132c945ac0ae8f76f149d66b1a9ffb3e6c45e1109e7f9488a6"} May 21 11:33:22.198235 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:22.198029 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-b7f67965d-272k2" event={"ID":"4540e5a0-576b-4722-acfc-a2cec978f9e3","Type":"ContainerStarted","Data":"e644e6b6ddf3da36cf3e4e5efa9770b3c78898987c71ec1de7f97f14aa1f2f88"} May 21 11:33:22.198235 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:22.198144 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-s79qs" May 21 11:33:22.233481 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:22.233437 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/router-default-b7f67965d-272k2" podStartSLOduration=33.233423436 podStartE2EDuration="33.233423436s" podCreationTimestamp="2026-05-21 11:32:49 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-21 11:33:22.233387097 +0000 UTC m=+157.128670642" watchObservedRunningTime="2026-05-21 11:33:22.233423436 +0000 UTC m=+157.128706980" May 21 11:33:23.197066 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:23.197028 2564 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-b7f67965d-272k2" May 21 11:33:23.199541 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:23.199518 2564 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-b7f67965d-272k2" May 21 11:33:23.200122 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:23.200104 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-b7f67965d-272k2" May 21 11:33:23.201061 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:23.201040 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-b7f67965d-272k2" May 21 11:33:23.692128 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:33:23.692019 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-xf5gc" podUID="414e12fe-b9ac-4bbf-8470-ad442215f5cb" May 21 11:33:23.736883 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:23.736847 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-7r4gg"] May 21 11:33:23.738802 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:23.738786 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-7r4gg" May 21 11:33:23.741811 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:23.741782 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" May 21 11:33:23.741943 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:23.741895 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-n5bbd\"" May 21 11:33:23.749604 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:23.749579 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-7r4gg"] May 21 11:33:23.923753 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:23.923721 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c856948d-3c99-4f33-966f-ab0a2f08e2cc-tls-certificates\") pod \"prometheus-operator-admission-webhook-6c4bbc67b7-7r4gg\" (UID: \"c856948d-3c99-4f33-966f-ab0a2f08e2cc\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-7r4gg" May 21 11:33:24.025112 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:24.025019 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c856948d-3c99-4f33-966f-ab0a2f08e2cc-tls-certificates\") pod \"prometheus-operator-admission-webhook-6c4bbc67b7-7r4gg\" (UID: \"c856948d-3c99-4f33-966f-ab0a2f08e2cc\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-7r4gg" May 21 11:33:24.027628 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:24.027598 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c856948d-3c99-4f33-966f-ab0a2f08e2cc-tls-certificates\") pod \"prometheus-operator-admission-webhook-6c4bbc67b7-7r4gg\" (UID: \"c856948d-3c99-4f33-966f-ab0a2f08e2cc\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-7r4gg" May 21 11:33:24.047427 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:24.047406 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-7r4gg" May 21 11:33:24.171943 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:24.171906 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-7r4gg"] May 21 11:33:24.175643 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:33:24.175608 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc856948d_3c99_4f33_966f_ab0a2f08e2cc.slice/crio-fcb9c7fc6285d8dd371efe2a20900c102f22cbc66c77bbeec6e66cdcef707694 WatchSource:0}: Error finding container fcb9c7fc6285d8dd371efe2a20900c102f22cbc66c77bbeec6e66cdcef707694: Status 404 returned error can't find the container with id fcb9c7fc6285d8dd371efe2a20900c102f22cbc66c77bbeec6e66cdcef707694 May 21 11:33:24.203353 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:24.203325 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-7r4gg" event={"ID":"c856948d-3c99-4f33-966f-ab0a2f08e2cc","Type":"ContainerStarted","Data":"fcb9c7fc6285d8dd371efe2a20900c102f22cbc66c77bbeec6e66cdcef707694"} May 21 11:33:26.209403 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:26.209363 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-7r4gg" event={"ID":"c856948d-3c99-4f33-966f-ab0a2f08e2cc","Type":"ContainerStarted","Data":"b3d585877cab9c93da67addbcaa59fabbba9fda7def31cc854fa58568fe03f01"} May 21 11:33:26.209815 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:26.209608 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-7r4gg" May 21 11:33:26.214403 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:26.214376 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-7r4gg" May 21 11:33:26.226417 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:26.226343 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-7r4gg" podStartSLOduration=2.222247053 podStartE2EDuration="3.226330094s" podCreationTimestamp="2026-05-21 11:33:23 +0000 UTC" firstStartedPulling="2026-05-21 11:33:24.177495784 +0000 UTC m=+159.072779318" lastFinishedPulling="2026-05-21 11:33:25.181578824 +0000 UTC m=+160.076862359" observedRunningTime="2026-05-21 11:33:26.225187225 +0000 UTC m=+161.120470771" watchObservedRunningTime="2026-05-21 11:33:26.226330094 +0000 UTC m=+161.121613639" May 21 11:33:26.804566 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:26.804538 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7779bbdd78-ltkmn"] May 21 11:33:26.826901 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:26.826872 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7779bbdd78-ltkmn"] May 21 11:33:26.827049 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:26.826922 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7779bbdd78-ltkmn" May 21 11:33:26.830701 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:26.830648 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" May 21 11:33:26.830961 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:26.830943 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" May 21 11:33:26.832204 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:26.832148 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" May 21 11:33:26.832299 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:26.832205 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" May 21 11:33:26.832299 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:26.832205 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-648j6\"" May 21 11:33:26.832299 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:26.832208 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" May 21 11:33:26.844858 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:26.844836 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0295ffc6-c516-4ec8-961e-3e7afcac0d6f-metrics-client-ca\") pod \"prometheus-operator-7779bbdd78-ltkmn\" (UID: \"0295ffc6-c516-4ec8-961e-3e7afcac0d6f\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-ltkmn" May 21 11:33:26.844988 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:26.844880 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27ktk\" (UniqueName: \"kubernetes.io/projected/0295ffc6-c516-4ec8-961e-3e7afcac0d6f-kube-api-access-27ktk\") pod \"prometheus-operator-7779bbdd78-ltkmn\" (UID: \"0295ffc6-c516-4ec8-961e-3e7afcac0d6f\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-ltkmn" May 21 11:33:26.844988 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:26.844978 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/0295ffc6-c516-4ec8-961e-3e7afcac0d6f-prometheus-operator-tls\") pod \"prometheus-operator-7779bbdd78-ltkmn\" (UID: \"0295ffc6-c516-4ec8-961e-3e7afcac0d6f\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-ltkmn" May 21 11:33:26.845136 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:26.845052 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0295ffc6-c516-4ec8-961e-3e7afcac0d6f-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7779bbdd78-ltkmn\" (UID: \"0295ffc6-c516-4ec8-961e-3e7afcac0d6f\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-ltkmn" May 21 11:33:26.946236 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:26.946202 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0295ffc6-c516-4ec8-961e-3e7afcac0d6f-metrics-client-ca\") pod \"prometheus-operator-7779bbdd78-ltkmn\" (UID: \"0295ffc6-c516-4ec8-961e-3e7afcac0d6f\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-ltkmn" May 21 11:33:26.946401 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:26.946243 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-27ktk\" (UniqueName: \"kubernetes.io/projected/0295ffc6-c516-4ec8-961e-3e7afcac0d6f-kube-api-access-27ktk\") pod \"prometheus-operator-7779bbdd78-ltkmn\" (UID: \"0295ffc6-c516-4ec8-961e-3e7afcac0d6f\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-ltkmn" May 21 11:33:26.946467 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:26.946396 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b14cbac4-0b39-404a-a413-a5ccf5fb6472-cert\") pod \"ingress-canary-s79qs\" (UID: \"b14cbac4-0b39-404a-a413-a5ccf5fb6472\") " pod="openshift-ingress-canary/ingress-canary-s79qs" May 21 11:33:26.946467 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:26.946452 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/0295ffc6-c516-4ec8-961e-3e7afcac0d6f-prometheus-operator-tls\") pod \"prometheus-operator-7779bbdd78-ltkmn\" (UID: \"0295ffc6-c516-4ec8-961e-3e7afcac0d6f\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-ltkmn" May 21 11:33:26.946588 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:26.946524 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2190640c-a631-4c98-8d97-ade39bb1567a-metrics-tls\") pod \"dns-default-2pfk6\" (UID: \"2190640c-a631-4c98-8d97-ade39bb1567a\") " pod="openshift-dns/dns-default-2pfk6" May 21 11:33:26.946588 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:26.946553 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0295ffc6-c516-4ec8-961e-3e7afcac0d6f-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7779bbdd78-ltkmn\" (UID: \"0295ffc6-c516-4ec8-961e-3e7afcac0d6f\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-ltkmn" May 21 11:33:26.946588 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:33:26.946567 2564 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found May 21 11:33:26.946737 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:33:26.946622 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0295ffc6-c516-4ec8-961e-3e7afcac0d6f-prometheus-operator-tls podName:0295ffc6-c516-4ec8-961e-3e7afcac0d6f nodeName:}" failed. No retries permitted until 2026-05-21 11:33:27.44660441 +0000 UTC m=+162.341887942 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/0295ffc6-c516-4ec8-961e-3e7afcac0d6f-prometheus-operator-tls") pod "prometheus-operator-7779bbdd78-ltkmn" (UID: "0295ffc6-c516-4ec8-961e-3e7afcac0d6f") : secret "prometheus-operator-tls" not found May 21 11:33:26.947118 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:26.947063 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0295ffc6-c516-4ec8-961e-3e7afcac0d6f-metrics-client-ca\") pod \"prometheus-operator-7779bbdd78-ltkmn\" (UID: \"0295ffc6-c516-4ec8-961e-3e7afcac0d6f\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-ltkmn" May 21 11:33:26.949158 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:26.949136 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b14cbac4-0b39-404a-a413-a5ccf5fb6472-cert\") pod \"ingress-canary-s79qs\" (UID: \"b14cbac4-0b39-404a-a413-a5ccf5fb6472\") " pod="openshift-ingress-canary/ingress-canary-s79qs" May 21 11:33:26.949250 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:26.949163 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2190640c-a631-4c98-8d97-ade39bb1567a-metrics-tls\") pod \"dns-default-2pfk6\" (UID: \"2190640c-a631-4c98-8d97-ade39bb1567a\") " pod="openshift-dns/dns-default-2pfk6" May 21 11:33:26.949250 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:26.949177 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0295ffc6-c516-4ec8-961e-3e7afcac0d6f-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7779bbdd78-ltkmn\" (UID: \"0295ffc6-c516-4ec8-961e-3e7afcac0d6f\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-ltkmn" May 21 11:33:26.957271 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:26.957244 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-27ktk\" (UniqueName: \"kubernetes.io/projected/0295ffc6-c516-4ec8-961e-3e7afcac0d6f-kube-api-access-27ktk\") pod \"prometheus-operator-7779bbdd78-ltkmn\" (UID: \"0295ffc6-c516-4ec8-961e-3e7afcac0d6f\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-ltkmn" May 21 11:33:27.003269 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:27.003241 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-q5l2g\"" May 21 11:33:27.003269 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:27.003245 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wrdr2\"" May 21 11:33:27.009021 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:27.009004 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-s79qs" May 21 11:33:27.009149 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:27.009132 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2pfk6" May 21 11:33:27.144187 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:27.144156 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2pfk6"] May 21 11:33:27.147435 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:33:27.147403 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2190640c_a631_4c98_8d97_ade39bb1567a.slice/crio-6f4dd88846072e883c9ff16d041dd0261ae96addf20e360ed224458158c36e98 WatchSource:0}: Error finding container 6f4dd88846072e883c9ff16d041dd0261ae96addf20e360ed224458158c36e98: Status 404 returned error can't find the container with id 6f4dd88846072e883c9ff16d041dd0261ae96addf20e360ed224458158c36e98 May 21 11:33:27.160471 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:27.160450 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-s79qs"] May 21 11:33:27.163816 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:33:27.163791 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb14cbac4_0b39_404a_a413_a5ccf5fb6472.slice/crio-916e1a97a68757c4cecdfb20cd865490fb4b894420b222248d5819a4f8859c37 WatchSource:0}: Error finding container 916e1a97a68757c4cecdfb20cd865490fb4b894420b222248d5819a4f8859c37: Status 404 returned error can't find the container with id 916e1a97a68757c4cecdfb20cd865490fb4b894420b222248d5819a4f8859c37 May 21 11:33:27.212589 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:27.212556 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-s79qs" event={"ID":"b14cbac4-0b39-404a-a413-a5ccf5fb6472","Type":"ContainerStarted","Data":"916e1a97a68757c4cecdfb20cd865490fb4b894420b222248d5819a4f8859c37"} May 21 11:33:27.213660 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:27.213632 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2pfk6" event={"ID":"2190640c-a631-4c98-8d97-ade39bb1567a","Type":"ContainerStarted","Data":"6f4dd88846072e883c9ff16d041dd0261ae96addf20e360ed224458158c36e98"} May 21 11:33:27.450199 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:27.450126 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/0295ffc6-c516-4ec8-961e-3e7afcac0d6f-prometheus-operator-tls\") pod \"prometheus-operator-7779bbdd78-ltkmn\" (UID: \"0295ffc6-c516-4ec8-961e-3e7afcac0d6f\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-ltkmn" May 21 11:33:27.452571 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:27.452541 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/0295ffc6-c516-4ec8-961e-3e7afcac0d6f-prometheus-operator-tls\") pod \"prometheus-operator-7779bbdd78-ltkmn\" (UID: \"0295ffc6-c516-4ec8-961e-3e7afcac0d6f\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-ltkmn" May 21 11:33:27.736540 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:27.736242 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7779bbdd78-ltkmn" May 21 11:33:27.885940 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:27.885902 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7779bbdd78-ltkmn"] May 21 11:33:27.890159 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:33:27.890127 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0295ffc6_c516_4ec8_961e_3e7afcac0d6f.slice/crio-ffed8f74aef27d435177d76c758fe7b261e229590caf9576af30c3a8e394a934 WatchSource:0}: Error finding container ffed8f74aef27d435177d76c758fe7b261e229590caf9576af30c3a8e394a934: Status 404 returned error can't find the container with id ffed8f74aef27d435177d76c758fe7b261e229590caf9576af30c3a8e394a934 May 21 11:33:28.218248 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:28.218199 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7779bbdd78-ltkmn" event={"ID":"0295ffc6-c516-4ec8-961e-3e7afcac0d6f","Type":"ContainerStarted","Data":"ffed8f74aef27d435177d76c758fe7b261e229590caf9576af30c3a8e394a934"} May 21 11:33:30.224565 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:30.224533 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-s79qs" event={"ID":"b14cbac4-0b39-404a-a413-a5ccf5fb6472","Type":"ContainerStarted","Data":"c28f6180ffcaaa1f5d374c115e58e5f420fc148fc2f8dd9906e8c3d7afc63c97"} May 21 11:33:30.226261 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:30.226234 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2pfk6" event={"ID":"2190640c-a631-4c98-8d97-ade39bb1567a","Type":"ContainerStarted","Data":"d952ee78f133b3517a3ea92cb550f0427c0db1abd9c8a276d077bd9f38946a21"} May 21 11:33:30.226399 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:30.226264 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2pfk6" event={"ID":"2190640c-a631-4c98-8d97-ade39bb1567a","Type":"ContainerStarted","Data":"f91cd70ba7a512cdefec4bdcd13420b9df4b108de0889366688e1fd75b8fbdc8"} May 21 11:33:30.226399 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:30.226368 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-2pfk6" May 21 11:33:30.227835 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:30.227813 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7779bbdd78-ltkmn" event={"ID":"0295ffc6-c516-4ec8-961e-3e7afcac0d6f","Type":"ContainerStarted","Data":"e4f8530b4ca4fa6c40f0519244a6a540e81f37e176bc90ed8219913b9e9abc4b"} May 21 11:33:30.228113 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:30.227838 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7779bbdd78-ltkmn" event={"ID":"0295ffc6-c516-4ec8-961e-3e7afcac0d6f","Type":"ContainerStarted","Data":"062e0f524944d42c3e5698b45f7112c9b25eafb01c753e88dceaf42759840c38"} May 21 11:33:30.242973 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:30.242933 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-s79qs" podStartSLOduration=129.880139971 podStartE2EDuration="2m12.242921249s" podCreationTimestamp="2026-05-21 11:31:18 +0000 UTC" firstStartedPulling="2026-05-21 11:33:27.165554321 +0000 UTC m=+162.060837844" lastFinishedPulling="2026-05-21 11:33:29.528335587 +0000 UTC m=+164.423619122" observedRunningTime="2026-05-21 11:33:30.24148599 +0000 UTC m=+165.136769548" watchObservedRunningTime="2026-05-21 11:33:30.242921249 +0000 UTC m=+165.138204793" May 21 11:33:30.265927 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:30.265876 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-2pfk6" podStartSLOduration=129.88451436 podStartE2EDuration="2m12.265862572s" podCreationTimestamp="2026-05-21 11:31:18 +0000 UTC" firstStartedPulling="2026-05-21 11:33:27.149249052 +0000 UTC m=+162.044532577" lastFinishedPulling="2026-05-21 11:33:29.530597252 +0000 UTC m=+164.425880789" observedRunningTime="2026-05-21 11:33:30.2647073 +0000 UTC m=+165.159990846" watchObservedRunningTime="2026-05-21 11:33:30.265862572 +0000 UTC m=+165.161146116" May 21 11:33:30.283563 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:30.283508 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7779bbdd78-ltkmn" podStartSLOduration=2.644584425 podStartE2EDuration="4.283494761s" podCreationTimestamp="2026-05-21 11:33:26 +0000 UTC" firstStartedPulling="2026-05-21 11:33:27.892636695 +0000 UTC m=+162.787920232" lastFinishedPulling="2026-05-21 11:33:29.531547039 +0000 UTC m=+164.426830568" observedRunningTime="2026-05-21 11:33:30.282708659 +0000 UTC m=+165.177992204" watchObservedRunningTime="2026-05-21 11:33:30.283494761 +0000 UTC m=+165.178778306" May 21 11:33:30.680071 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:30.680033 2564 scope.go:117] "RemoveContainer" containerID="f06f3aafcc7dec3728eaad07da827eb505ef43d0d3542e47d5fd014562d0d7da" May 21 11:33:30.680279 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:33:30.680261 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-76dff48d59-6g9xk_openshift-console-operator(be463222-9e99-47cb-bf6b-525913ca9e29)\"" pod="openshift-console-operator/console-operator-76dff48d59-6g9xk" podUID="be463222-9e99-47cb-bf6b-525913ca9e29" May 21 11:33:32.135203 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:32.135163 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-9c959559f-5m46j"] May 21 11:33:32.137422 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:32.137403 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-9c959559f-5m46j" May 21 11:33:32.140374 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:32.140349 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-c7948\"" May 21 11:33:32.140514 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:32.140412 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" May 21 11:33:32.140514 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:32.140357 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" May 21 11:33:32.148418 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:32.148393 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-9c959559f-5m46j"] May 21 11:33:32.156714 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:32.156691 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-lc6jm"] May 21 11:33:32.159255 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:32.159235 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-lc6jm" May 21 11:33:32.161986 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:32.161961 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" May 21 11:33:32.162155 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:32.162007 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" May 21 11:33:32.162503 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:32.162465 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" May 21 11:33:32.162629 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:32.162608 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-5qmnx\"" May 21 11:33:32.190638 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:32.190607 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d5981a1d-940e-4b8c-b48b-5db4ba823683-node-exporter-tls\") pod \"node-exporter-lc6jm\" (UID: \"d5981a1d-940e-4b8c-b48b-5db4ba823683\") " pod="openshift-monitoring/node-exporter-lc6jm" May 21 11:33:32.190810 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:32.190645 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d5981a1d-940e-4b8c-b48b-5db4ba823683-sys\") pod \"node-exporter-lc6jm\" (UID: \"d5981a1d-940e-4b8c-b48b-5db4ba823683\") " pod="openshift-monitoring/node-exporter-lc6jm" May 21 11:33:32.190810 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:32.190671 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d5981a1d-940e-4b8c-b48b-5db4ba823683-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lc6jm\" (UID: \"d5981a1d-940e-4b8c-b48b-5db4ba823683\") " pod="openshift-monitoring/node-exporter-lc6jm" May 21 11:33:32.190810 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:32.190714 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qs2l5\" (UniqueName: \"kubernetes.io/projected/3f352f56-2ade-4408-98eb-0384957496ef-kube-api-access-qs2l5\") pod \"openshift-state-metrics-9c959559f-5m46j\" (UID: \"3f352f56-2ade-4408-98eb-0384957496ef\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-5m46j" May 21 11:33:32.190810 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:32.190790 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d5981a1d-940e-4b8c-b48b-5db4ba823683-metrics-client-ca\") pod \"node-exporter-lc6jm\" (UID: \"d5981a1d-940e-4b8c-b48b-5db4ba823683\") " pod="openshift-monitoring/node-exporter-lc6jm" May 21 11:33:32.190963 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:32.190811 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3f352f56-2ade-4408-98eb-0384957496ef-openshift-state-metrics-tls\") pod \"openshift-state-metrics-9c959559f-5m46j\" (UID: \"3f352f56-2ade-4408-98eb-0384957496ef\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-5m46j" May 21 11:33:32.190963 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:32.190835 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3f352f56-2ade-4408-98eb-0384957496ef-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-9c959559f-5m46j\" (UID: \"3f352f56-2ade-4408-98eb-0384957496ef\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-5m46j" May 21 11:33:32.190963 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:32.190889 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3f352f56-2ade-4408-98eb-0384957496ef-metrics-client-ca\") pod \"openshift-state-metrics-9c959559f-5m46j\" (UID: \"3f352f56-2ade-4408-98eb-0384957496ef\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-5m46j" May 21 11:33:32.190963 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:32.190929 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d5981a1d-940e-4b8c-b48b-5db4ba823683-root\") pod \"node-exporter-lc6jm\" (UID: \"d5981a1d-940e-4b8c-b48b-5db4ba823683\") " pod="openshift-monitoring/node-exporter-lc6jm" May 21 11:33:32.190963 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:32.190951 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d5981a1d-940e-4b8c-b48b-5db4ba823683-node-exporter-accelerators-collector-config\") pod \"node-exporter-lc6jm\" (UID: \"d5981a1d-940e-4b8c-b48b-5db4ba823683\") " pod="openshift-monitoring/node-exporter-lc6jm" May 21 11:33:32.191214 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:32.190973 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d5981a1d-940e-4b8c-b48b-5db4ba823683-node-exporter-wtmp\") pod \"node-exporter-lc6jm\" (UID: \"d5981a1d-940e-4b8c-b48b-5db4ba823683\") " pod="openshift-monitoring/node-exporter-lc6jm" May 21 11:33:32.191214 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:32.191019 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgzng\" (UniqueName: \"kubernetes.io/projected/d5981a1d-940e-4b8c-b48b-5db4ba823683-kube-api-access-wgzng\") pod \"node-exporter-lc6jm\" (UID: \"d5981a1d-940e-4b8c-b48b-5db4ba823683\") " pod="openshift-monitoring/node-exporter-lc6jm" May 21 11:33:32.191214 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:32.191050 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d5981a1d-940e-4b8c-b48b-5db4ba823683-node-exporter-textfile\") pod \"node-exporter-lc6jm\" (UID: \"d5981a1d-940e-4b8c-b48b-5db4ba823683\") " pod="openshift-monitoring/node-exporter-lc6jm" May 21 11:33:32.292177 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:32.292144 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d5981a1d-940e-4b8c-b48b-5db4ba823683-node-exporter-textfile\") pod \"node-exporter-lc6jm\" (UID: \"d5981a1d-940e-4b8c-b48b-5db4ba823683\") " pod="openshift-monitoring/node-exporter-lc6jm" May 21 11:33:32.292349 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:32.292189 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d5981a1d-940e-4b8c-b48b-5db4ba823683-node-exporter-tls\") pod \"node-exporter-lc6jm\" (UID: \"d5981a1d-940e-4b8c-b48b-5db4ba823683\") " pod="openshift-monitoring/node-exporter-lc6jm" May 21 11:33:32.292349 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:32.292220 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d5981a1d-940e-4b8c-b48b-5db4ba823683-sys\") pod \"node-exporter-lc6jm\" (UID: \"d5981a1d-940e-4b8c-b48b-5db4ba823683\") " pod="openshift-monitoring/node-exporter-lc6jm" May 21 11:33:32.292349 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:32.292245 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d5981a1d-940e-4b8c-b48b-5db4ba823683-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lc6jm\" (UID: \"d5981a1d-940e-4b8c-b48b-5db4ba823683\") " pod="openshift-monitoring/node-exporter-lc6jm" May 21 11:33:32.292349 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:32.292272 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qs2l5\" (UniqueName: \"kubernetes.io/projected/3f352f56-2ade-4408-98eb-0384957496ef-kube-api-access-qs2l5\") pod \"openshift-state-metrics-9c959559f-5m46j\" (UID: \"3f352f56-2ade-4408-98eb-0384957496ef\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-5m46j" May 21 11:33:32.292349 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:32.292309 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d5981a1d-940e-4b8c-b48b-5db4ba823683-metrics-client-ca\") pod \"node-exporter-lc6jm\" (UID: \"d5981a1d-940e-4b8c-b48b-5db4ba823683\") " pod="openshift-monitoring/node-exporter-lc6jm" May 21 11:33:32.292349 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:32.292333 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3f352f56-2ade-4408-98eb-0384957496ef-openshift-state-metrics-tls\") pod \"openshift-state-metrics-9c959559f-5m46j\" (UID: \"3f352f56-2ade-4408-98eb-0384957496ef\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-5m46j" May 21 11:33:32.292647 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:32.292350 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d5981a1d-940e-4b8c-b48b-5db4ba823683-sys\") pod \"node-exporter-lc6jm\" (UID: \"d5981a1d-940e-4b8c-b48b-5db4ba823683\") " pod="openshift-monitoring/node-exporter-lc6jm" May 21 11:33:32.292647 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:32.292359 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3f352f56-2ade-4408-98eb-0384957496ef-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-9c959559f-5m46j\" (UID: \"3f352f56-2ade-4408-98eb-0384957496ef\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-5m46j" May 21 11:33:32.292647 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:32.292435 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3f352f56-2ade-4408-98eb-0384957496ef-metrics-client-ca\") pod \"openshift-state-metrics-9c959559f-5m46j\" (UID: \"3f352f56-2ade-4408-98eb-0384957496ef\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-5m46j" May 21 11:33:32.292647 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:32.292479 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d5981a1d-940e-4b8c-b48b-5db4ba823683-root\") pod \"node-exporter-lc6jm\" (UID: \"d5981a1d-940e-4b8c-b48b-5db4ba823683\") " pod="openshift-monitoring/node-exporter-lc6jm" May 21 11:33:32.292647 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:32.292517 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d5981a1d-940e-4b8c-b48b-5db4ba823683-node-exporter-accelerators-collector-config\") pod \"node-exporter-lc6jm\" (UID: \"d5981a1d-940e-4b8c-b48b-5db4ba823683\") " pod="openshift-monitoring/node-exporter-lc6jm" May 21 11:33:32.292647 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:33:32.292530 2564 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found May 21 11:33:32.292647 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:32.292535 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d5981a1d-940e-4b8c-b48b-5db4ba823683-node-exporter-textfile\") pod \"node-exporter-lc6jm\" (UID: \"d5981a1d-940e-4b8c-b48b-5db4ba823683\") " pod="openshift-monitoring/node-exporter-lc6jm" May 21 11:33:32.292647 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:32.292553 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d5981a1d-940e-4b8c-b48b-5db4ba823683-node-exporter-wtmp\") pod \"node-exporter-lc6jm\" (UID: \"d5981a1d-940e-4b8c-b48b-5db4ba823683\") " pod="openshift-monitoring/node-exporter-lc6jm" May 21 11:33:32.292647 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:32.292580 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wgzng\" (UniqueName: \"kubernetes.io/projected/d5981a1d-940e-4b8c-b48b-5db4ba823683-kube-api-access-wgzng\") pod \"node-exporter-lc6jm\" (UID: \"d5981a1d-940e-4b8c-b48b-5db4ba823683\") " pod="openshift-monitoring/node-exporter-lc6jm" May 21 11:33:32.292647 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:33:32.292598 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f352f56-2ade-4408-98eb-0384957496ef-openshift-state-metrics-tls podName:3f352f56-2ade-4408-98eb-0384957496ef nodeName:}" failed. No retries permitted until 2026-05-21 11:33:32.792576036 +0000 UTC m=+167.687859572 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/3f352f56-2ade-4408-98eb-0384957496ef-openshift-state-metrics-tls") pod "openshift-state-metrics-9c959559f-5m46j" (UID: "3f352f56-2ade-4408-98eb-0384957496ef") : secret "openshift-state-metrics-tls" not found May 21 11:33:32.293122 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:32.292784 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d5981a1d-940e-4b8c-b48b-5db4ba823683-root\") pod \"node-exporter-lc6jm\" (UID: \"d5981a1d-940e-4b8c-b48b-5db4ba823683\") " pod="openshift-monitoring/node-exporter-lc6jm" May 21 11:33:32.293122 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:32.292993 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d5981a1d-940e-4b8c-b48b-5db4ba823683-node-exporter-wtmp\") pod \"node-exporter-lc6jm\" (UID: \"d5981a1d-940e-4b8c-b48b-5db4ba823683\") " pod="openshift-monitoring/node-exporter-lc6jm" May 21 11:33:32.293225 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:32.293167 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d5981a1d-940e-4b8c-b48b-5db4ba823683-metrics-client-ca\") pod \"node-exporter-lc6jm\" (UID: \"d5981a1d-940e-4b8c-b48b-5db4ba823683\") " pod="openshift-monitoring/node-exporter-lc6jm" May 21 11:33:32.293307 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:32.293277 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3f352f56-2ade-4408-98eb-0384957496ef-metrics-client-ca\") pod \"openshift-state-metrics-9c959559f-5m46j\" (UID: \"3f352f56-2ade-4408-98eb-0384957496ef\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-5m46j" May 21 11:33:32.293307 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:32.293288 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d5981a1d-940e-4b8c-b48b-5db4ba823683-node-exporter-accelerators-collector-config\") pod \"node-exporter-lc6jm\" (UID: \"d5981a1d-940e-4b8c-b48b-5db4ba823683\") " pod="openshift-monitoring/node-exporter-lc6jm" May 21 11:33:32.295371 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:32.295339 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d5981a1d-940e-4b8c-b48b-5db4ba823683-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lc6jm\" (UID: \"d5981a1d-940e-4b8c-b48b-5db4ba823683\") " pod="openshift-monitoring/node-exporter-lc6jm" May 21 11:33:32.295523 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:32.295500 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3f352f56-2ade-4408-98eb-0384957496ef-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-9c959559f-5m46j\" (UID: \"3f352f56-2ade-4408-98eb-0384957496ef\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-5m46j" May 21 11:33:32.295603 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:32.295588 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d5981a1d-940e-4b8c-b48b-5db4ba823683-node-exporter-tls\") pod \"node-exporter-lc6jm\" (UID: \"d5981a1d-940e-4b8c-b48b-5db4ba823683\") " pod="openshift-monitoring/node-exporter-lc6jm" May 21 11:33:32.301706 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:32.301667 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgzng\" (UniqueName: \"kubernetes.io/projected/d5981a1d-940e-4b8c-b48b-5db4ba823683-kube-api-access-wgzng\") pod \"node-exporter-lc6jm\" (UID: \"d5981a1d-940e-4b8c-b48b-5db4ba823683\") " pod="openshift-monitoring/node-exporter-lc6jm" May 21 11:33:32.302685 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:32.302666 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qs2l5\" (UniqueName: \"kubernetes.io/projected/3f352f56-2ade-4408-98eb-0384957496ef-kube-api-access-qs2l5\") pod \"openshift-state-metrics-9c959559f-5m46j\" (UID: \"3f352f56-2ade-4408-98eb-0384957496ef\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-5m46j" May 21 11:33:32.470996 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:32.470901 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-lc6jm" May 21 11:33:32.479699 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:33:32.479662 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5981a1d_940e_4b8c_b48b_5db4ba823683.slice/crio-3d1a0a74ab0719504d52bd31634d6cae0b6c0ff899fd9ab8c8c2a6b0c2c130de WatchSource:0}: Error finding container 3d1a0a74ab0719504d52bd31634d6cae0b6c0ff899fd9ab8c8c2a6b0c2c130de: Status 404 returned error can't find the container with id 3d1a0a74ab0719504d52bd31634d6cae0b6c0ff899fd9ab8c8c2a6b0c2c130de May 21 11:33:32.797204 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:32.797178 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3f352f56-2ade-4408-98eb-0384957496ef-openshift-state-metrics-tls\") pod \"openshift-state-metrics-9c959559f-5m46j\" (UID: \"3f352f56-2ade-4408-98eb-0384957496ef\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-5m46j" May 21 11:33:32.799666 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:32.799637 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3f352f56-2ade-4408-98eb-0384957496ef-openshift-state-metrics-tls\") pod \"openshift-state-metrics-9c959559f-5m46j\" (UID: \"3f352f56-2ade-4408-98eb-0384957496ef\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-5m46j" May 21 11:33:33.049302 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:33.049221 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-9c959559f-5m46j" May 21 11:33:33.208187 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:33.208148 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-9c959559f-5m46j"] May 21 11:33:33.212755 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:33:33.212725 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f352f56_2ade_4408_98eb_0384957496ef.slice/crio-ee8ace95eaaac1aa55b43521d7e0b1225ad9c850549fe65a944743294fd35259 WatchSource:0}: Error finding container ee8ace95eaaac1aa55b43521d7e0b1225ad9c850549fe65a944743294fd35259: Status 404 returned error can't find the container with id ee8ace95eaaac1aa55b43521d7e0b1225ad9c850549fe65a944743294fd35259 May 21 11:33:33.239659 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:33.238795 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] May 21 11:33:33.242276 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:33.242242 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lc6jm" event={"ID":"d5981a1d-940e-4b8c-b48b-5db4ba823683","Type":"ContainerStarted","Data":"3d1a0a74ab0719504d52bd31634d6cae0b6c0ff899fd9ab8c8c2a6b0c2c130de"} May 21 11:33:33.242653 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:33.242427 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" May 21 11:33:33.245152 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:33.245120 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-9c959559f-5m46j" event={"ID":"3f352f56-2ade-4408-98eb-0384957496ef","Type":"ContainerStarted","Data":"ee8ace95eaaac1aa55b43521d7e0b1225ad9c850549fe65a944743294fd35259"} May 21 11:33:33.245736 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:33.245714 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" May 21 11:33:33.245736 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:33.245727 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" May 21 11:33:33.245898 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:33.245867 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-kpmvs\"" May 21 11:33:33.246000 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:33.245906 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" May 21 11:33:33.246000 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:33.245984 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" May 21 11:33:33.246217 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:33.246195 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" May 21 11:33:33.248347 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:33.247168 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" May 21 11:33:33.248347 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:33.247515 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" May 21 11:33:33.248347 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:33.247722 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" May 21 11:33:33.248347 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:33.247775 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" May 21 11:33:33.264597 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:33.263303 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] May 21 11:33:33.306191 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:33.306036 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:33:33.306191 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:33.306100 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-web-config\") pod \"alertmanager-main-0\" (UID: \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:33:33.306191 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:33.306135 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-config-volume\") pod \"alertmanager-main-0\" (UID: \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:33:33.306463 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:33.306349 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:33:33.306463 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:33.306451 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:33:33.306567 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:33.306524 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:33:33.306675 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:33.306654 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-config-out\") pod \"alertmanager-main-0\" (UID: \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:33:33.306736 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:33.306688 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:33:33.306736 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:33.306722 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xm7kt\" (UniqueName: \"kubernetes.io/projected/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-kube-api-access-xm7kt\") pod \"alertmanager-main-0\" (UID: \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:33:33.306824 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:33.306772 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:33:33.306824 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:33.306807 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:33:33.306923 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:33.306832 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:33:33.306976 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:33.306915 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:33:33.407658 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:33.407618 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:33:33.407847 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:33.407675 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-web-config\") pod \"alertmanager-main-0\" (UID: \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:33:33.407847 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:33.407704 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-config-volume\") pod \"alertmanager-main-0\" (UID: \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:33:33.407847 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:33.407726 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:33:33.407847 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:33.407823 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:33:33.408030 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:33.407859 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:33:33.408030 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:33.407934 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-config-out\") pod \"alertmanager-main-0\" (UID: \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:33:33.408030 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:33.407968 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:33:33.408030 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:33.407999 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xm7kt\" (UniqueName: \"kubernetes.io/projected/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-kube-api-access-xm7kt\") pod \"alertmanager-main-0\" (UID: \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:33:33.408255 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:33:33.408035 2564 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found May 21 11:33:33.408255 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:33.408042 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:33:33.408255 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:33:33.408146 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-secret-alertmanager-main-tls podName:9ee6d1da-97d3-4f22-896c-b1d8ade178bb nodeName:}" failed. No retries permitted until 2026-05-21 11:33:33.908122279 +0000 UTC m=+168.803405816 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "9ee6d1da-97d3-4f22-896c-b1d8ade178bb") : secret "alertmanager-main-tls" not found May 21 11:33:33.408255 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:33.408131 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:33:33.408255 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:33.408196 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:33:33.408255 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:33.408242 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:33:33.409417 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:33.409162 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:33:33.409417 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:33:33.409278 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-alertmanager-trusted-ca-bundle podName:9ee6d1da-97d3-4f22-896c-b1d8ade178bb nodeName:}" failed. No retries permitted until 2026-05-21 11:33:33.909261523 +0000 UTC m=+168.804545047 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "9ee6d1da-97d3-4f22-896c-b1d8ade178bb") : configmap references non-existent config key: ca-bundle.crt May 21 11:33:33.409615 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:33.409493 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:33:33.412256 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:33.412208 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-web-config\") pod \"alertmanager-main-0\" (UID: \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:33:33.412782 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:33.412708 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:33:33.413270 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:33.412951 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:33:33.413270 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:33.413203 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-config-out\") pod \"alertmanager-main-0\" (UID: \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:33:33.413440 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:33.413415 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:33:33.413637 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:33.413616 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:33:33.414198 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:33.414106 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-config-volume\") pod \"alertmanager-main-0\" (UID: \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:33:33.415412 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:33.415381 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:33:33.417686 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:33.417665 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xm7kt\" (UniqueName: \"kubernetes.io/projected/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-kube-api-access-xm7kt\") pod \"alertmanager-main-0\" (UID: \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:33:33.912286 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:33.912199 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:33:33.912286 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:33.912271 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:33:33.913167 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:33.913146 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:33:33.914919 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:33.914891 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:33:34.166784 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:34.166696 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-67f6f86555-zx427"] May 21 11:33:34.169794 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:34.169768 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" May 21 11:33:34.169981 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:34.169959 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-67f6f86555-zx427" May 21 11:33:34.173971 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:34.173947 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" May 21 11:33:34.174113 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:34.173950 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" May 21 11:33:34.174113 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:34.174069 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-3o1n9fkevhpok\"" May 21 11:33:34.174373 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:34.174354 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-9572m\"" May 21 11:33:34.174818 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:34.174801 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" May 21 11:33:34.175646 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:34.175128 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" May 21 11:33:34.178827 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:34.178603 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" May 21 11:33:34.200679 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:34.200648 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-67f6f86555-zx427"] May 21 11:33:34.216692 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:34.216594 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/473c66ed-1f18-4de7-a9f3-34be157198ec-secret-grpc-tls\") pod \"thanos-querier-67f6f86555-zx427\" (UID: \"473c66ed-1f18-4de7-a9f3-34be157198ec\") " pod="openshift-monitoring/thanos-querier-67f6f86555-zx427" May 21 11:33:34.216692 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:34.216677 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/473c66ed-1f18-4de7-a9f3-34be157198ec-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-67f6f86555-zx427\" (UID: \"473c66ed-1f18-4de7-a9f3-34be157198ec\") " pod="openshift-monitoring/thanos-querier-67f6f86555-zx427" May 21 11:33:34.217278 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:34.216720 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/473c66ed-1f18-4de7-a9f3-34be157198ec-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-67f6f86555-zx427\" (UID: \"473c66ed-1f18-4de7-a9f3-34be157198ec\") " pod="openshift-monitoring/thanos-querier-67f6f86555-zx427" May 21 11:33:34.217278 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:34.216752 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/473c66ed-1f18-4de7-a9f3-34be157198ec-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-67f6f86555-zx427\" (UID: \"473c66ed-1f18-4de7-a9f3-34be157198ec\") " pod="openshift-monitoring/thanos-querier-67f6f86555-zx427" May 21 11:33:34.217278 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:34.216796 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/473c66ed-1f18-4de7-a9f3-34be157198ec-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-67f6f86555-zx427\" (UID: \"473c66ed-1f18-4de7-a9f3-34be157198ec\") " pod="openshift-monitoring/thanos-querier-67f6f86555-zx427" May 21 11:33:34.217278 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:34.216864 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/473c66ed-1f18-4de7-a9f3-34be157198ec-secret-thanos-querier-tls\") pod \"thanos-querier-67f6f86555-zx427\" (UID: \"473c66ed-1f18-4de7-a9f3-34be157198ec\") " pod="openshift-monitoring/thanos-querier-67f6f86555-zx427" May 21 11:33:34.217278 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:34.216897 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/473c66ed-1f18-4de7-a9f3-34be157198ec-metrics-client-ca\") pod \"thanos-querier-67f6f86555-zx427\" (UID: \"473c66ed-1f18-4de7-a9f3-34be157198ec\") " pod="openshift-monitoring/thanos-querier-67f6f86555-zx427" May 21 11:33:34.217278 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:34.216927 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svwkp\" (UniqueName: \"kubernetes.io/projected/473c66ed-1f18-4de7-a9f3-34be157198ec-kube-api-access-svwkp\") pod \"thanos-querier-67f6f86555-zx427\" (UID: \"473c66ed-1f18-4de7-a9f3-34be157198ec\") " pod="openshift-monitoring/thanos-querier-67f6f86555-zx427" May 21 11:33:34.250294 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:34.250243 2564 generic.go:358] "Generic (PLEG): container finished" podID="d5981a1d-940e-4b8c-b48b-5db4ba823683" containerID="0cbb99e68eb2b8f9580a70591353942cdd8c59861ededf705aee92b6c1eff855" exitCode=0 May 21 11:33:34.250437 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:34.250360 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lc6jm" event={"ID":"d5981a1d-940e-4b8c-b48b-5db4ba823683","Type":"ContainerDied","Data":"0cbb99e68eb2b8f9580a70591353942cdd8c59861ededf705aee92b6c1eff855"} May 21 11:33:34.252372 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:34.252346 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-9c959559f-5m46j" event={"ID":"3f352f56-2ade-4408-98eb-0384957496ef","Type":"ContainerStarted","Data":"9ccbfab9d02d8e3f84c32ef038fbffd3335b65d073dd6c9dc276e811ee4e3a06"} May 21 11:33:34.252456 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:34.252378 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-9c959559f-5m46j" event={"ID":"3f352f56-2ade-4408-98eb-0384957496ef","Type":"ContainerStarted","Data":"f5c8ce3e4a184ddaf99f085519f9ee0f7fcb4dabf44d667cca0e24cc562160bf"} May 21 11:33:34.317947 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:34.317914 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/473c66ed-1f18-4de7-a9f3-34be157198ec-metrics-client-ca\") pod \"thanos-querier-67f6f86555-zx427\" (UID: \"473c66ed-1f18-4de7-a9f3-34be157198ec\") " pod="openshift-monitoring/thanos-querier-67f6f86555-zx427" May 21 11:33:34.318123 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:34.317987 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-svwkp\" (UniqueName: \"kubernetes.io/projected/473c66ed-1f18-4de7-a9f3-34be157198ec-kube-api-access-svwkp\") pod \"thanos-querier-67f6f86555-zx427\" (UID: \"473c66ed-1f18-4de7-a9f3-34be157198ec\") " pod="openshift-monitoring/thanos-querier-67f6f86555-zx427" May 21 11:33:34.318123 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:34.318017 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/473c66ed-1f18-4de7-a9f3-34be157198ec-secret-grpc-tls\") pod \"thanos-querier-67f6f86555-zx427\" (UID: \"473c66ed-1f18-4de7-a9f3-34be157198ec\") " pod="openshift-monitoring/thanos-querier-67f6f86555-zx427" May 21 11:33:34.318246 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:34.318121 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/473c66ed-1f18-4de7-a9f3-34be157198ec-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-67f6f86555-zx427\" (UID: \"473c66ed-1f18-4de7-a9f3-34be157198ec\") " pod="openshift-monitoring/thanos-querier-67f6f86555-zx427" May 21 11:33:34.318246 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:34.318180 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/473c66ed-1f18-4de7-a9f3-34be157198ec-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-67f6f86555-zx427\" (UID: \"473c66ed-1f18-4de7-a9f3-34be157198ec\") " pod="openshift-monitoring/thanos-querier-67f6f86555-zx427" May 21 11:33:34.318246 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:34.318210 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/473c66ed-1f18-4de7-a9f3-34be157198ec-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-67f6f86555-zx427\" (UID: \"473c66ed-1f18-4de7-a9f3-34be157198ec\") " pod="openshift-monitoring/thanos-querier-67f6f86555-zx427" May 21 11:33:34.318355 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:34.318270 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/473c66ed-1f18-4de7-a9f3-34be157198ec-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-67f6f86555-zx427\" (UID: \"473c66ed-1f18-4de7-a9f3-34be157198ec\") " pod="openshift-monitoring/thanos-querier-67f6f86555-zx427" May 21 11:33:34.318420 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:34.318381 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/473c66ed-1f18-4de7-a9f3-34be157198ec-secret-thanos-querier-tls\") pod \"thanos-querier-67f6f86555-zx427\" (UID: \"473c66ed-1f18-4de7-a9f3-34be157198ec\") " pod="openshift-monitoring/thanos-querier-67f6f86555-zx427" May 21 11:33:34.319163 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:34.318756 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/473c66ed-1f18-4de7-a9f3-34be157198ec-metrics-client-ca\") pod \"thanos-querier-67f6f86555-zx427\" (UID: \"473c66ed-1f18-4de7-a9f3-34be157198ec\") " pod="openshift-monitoring/thanos-querier-67f6f86555-zx427" May 21 11:33:34.321428 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:34.321385 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/473c66ed-1f18-4de7-a9f3-34be157198ec-secret-thanos-querier-tls\") pod \"thanos-querier-67f6f86555-zx427\" (UID: \"473c66ed-1f18-4de7-a9f3-34be157198ec\") " pod="openshift-monitoring/thanos-querier-67f6f86555-zx427" May 21 11:33:34.321584 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:34.321559 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/473c66ed-1f18-4de7-a9f3-34be157198ec-secret-grpc-tls\") pod \"thanos-querier-67f6f86555-zx427\" (UID: \"473c66ed-1f18-4de7-a9f3-34be157198ec\") " pod="openshift-monitoring/thanos-querier-67f6f86555-zx427" May 21 11:33:34.321672 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:34.321652 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/473c66ed-1f18-4de7-a9f3-34be157198ec-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-67f6f86555-zx427\" (UID: \"473c66ed-1f18-4de7-a9f3-34be157198ec\") " pod="openshift-monitoring/thanos-querier-67f6f86555-zx427" May 21 11:33:34.321765 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:34.321746 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/473c66ed-1f18-4de7-a9f3-34be157198ec-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-67f6f86555-zx427\" (UID: \"473c66ed-1f18-4de7-a9f3-34be157198ec\") " pod="openshift-monitoring/thanos-querier-67f6f86555-zx427" May 21 11:33:34.321827 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:34.321771 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/473c66ed-1f18-4de7-a9f3-34be157198ec-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-67f6f86555-zx427\" (UID: \"473c66ed-1f18-4de7-a9f3-34be157198ec\") " pod="openshift-monitoring/thanos-querier-67f6f86555-zx427" May 21 11:33:34.322885 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:34.322867 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/473c66ed-1f18-4de7-a9f3-34be157198ec-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-67f6f86555-zx427\" (UID: \"473c66ed-1f18-4de7-a9f3-34be157198ec\") " pod="openshift-monitoring/thanos-querier-67f6f86555-zx427" May 21 11:33:34.327543 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:34.327429 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-svwkp\" (UniqueName: \"kubernetes.io/projected/473c66ed-1f18-4de7-a9f3-34be157198ec-kube-api-access-svwkp\") pod \"thanos-querier-67f6f86555-zx427\" (UID: \"473c66ed-1f18-4de7-a9f3-34be157198ec\") " pod="openshift-monitoring/thanos-querier-67f6f86555-zx427" May 21 11:33:34.330333 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:34.330312 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] May 21 11:33:34.474885 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:33:34.474699 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ee6d1da_97d3_4f22_896c_b1d8ade178bb.slice/crio-f46d14a8cac54049516fa3e2709b472821161c6857ae734c9839aeff3824f13b WatchSource:0}: Error finding container f46d14a8cac54049516fa3e2709b472821161c6857ae734c9839aeff3824f13b: Status 404 returned error can't find the container with id f46d14a8cac54049516fa3e2709b472821161c6857ae734c9839aeff3824f13b May 21 11:33:34.482606 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:34.482582 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-67f6f86555-zx427" May 21 11:33:34.672886 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:34.672759 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-67f6f86555-zx427"] May 21 11:33:34.675383 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:33:34.675348 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod473c66ed_1f18_4de7_a9f3_34be157198ec.slice/crio-5b179179aac2a01e3b3fbbf19dc50ba4f2b81d5e28311bfb811c07beaa74e6b2 WatchSource:0}: Error finding container 5b179179aac2a01e3b3fbbf19dc50ba4f2b81d5e28311bfb811c07beaa74e6b2: Status 404 returned error can't find the container with id 5b179179aac2a01e3b3fbbf19dc50ba4f2b81d5e28311bfb811c07beaa74e6b2 May 21 11:33:35.257522 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:35.257461 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9ee6d1da-97d3-4f22-896c-b1d8ade178bb","Type":"ContainerStarted","Data":"f46d14a8cac54049516fa3e2709b472821161c6857ae734c9839aeff3824f13b"} May 21 11:33:35.259779 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:35.259747 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-9c959559f-5m46j" event={"ID":"3f352f56-2ade-4408-98eb-0384957496ef","Type":"ContainerStarted","Data":"062859fc1a33897ae8661d8474af50cc8ca7c7c06e447c4102c6a30f278cdd42"} May 21 11:33:35.262577 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:35.262538 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lc6jm" event={"ID":"d5981a1d-940e-4b8c-b48b-5db4ba823683","Type":"ContainerStarted","Data":"f6c6bda5da525be156a1d4f6ad5c11a4283f8f7ac2ab9f8b0d289e7cf24e0857"} May 21 11:33:35.262688 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:35.262583 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lc6jm" event={"ID":"d5981a1d-940e-4b8c-b48b-5db4ba823683","Type":"ContainerStarted","Data":"5518d13873f51d5505b8910280d8d96761872530a0dde4e7d1c47a77a316820d"} May 21 11:33:35.264599 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:35.264498 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-67f6f86555-zx427" event={"ID":"473c66ed-1f18-4de7-a9f3-34be157198ec","Type":"ContainerStarted","Data":"5b179179aac2a01e3b3fbbf19dc50ba4f2b81d5e28311bfb811c07beaa74e6b2"} May 21 11:33:35.298786 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:35.298728 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-9c959559f-5m46j" podStartSLOduration=2.368220144 podStartE2EDuration="3.298707559s" podCreationTimestamp="2026-05-21 11:33:32 +0000 UTC" firstStartedPulling="2026-05-21 11:33:33.573252098 +0000 UTC m=+168.468535631" lastFinishedPulling="2026-05-21 11:33:34.50373952 +0000 UTC m=+169.399023046" observedRunningTime="2026-05-21 11:33:35.29481574 +0000 UTC m=+170.190099287" watchObservedRunningTime="2026-05-21 11:33:35.298707559 +0000 UTC m=+170.193991106" May 21 11:33:35.379128 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:35.379040 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-lc6jm" podStartSLOduration=2.317917716 podStartE2EDuration="3.379017491s" podCreationTimestamp="2026-05-21 11:33:32 +0000 UTC" firstStartedPulling="2026-05-21 11:33:32.481794325 +0000 UTC m=+167.377077851" lastFinishedPulling="2026-05-21 11:33:33.542894092 +0000 UTC m=+168.438177626" observedRunningTime="2026-05-21 11:33:35.377721465 +0000 UTC m=+170.273005011" watchObservedRunningTime="2026-05-21 11:33:35.379017491 +0000 UTC m=+170.274301037" May 21 11:33:36.269005 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:36.268965 2564 generic.go:358] "Generic (PLEG): container finished" podID="9ee6d1da-97d3-4f22-896c-b1d8ade178bb" containerID="027f7a51458344f5bf563523c8bac523aa43701896513c6ad2176119037703f2" exitCode=0 May 21 11:33:36.269791 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:36.269090 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9ee6d1da-97d3-4f22-896c-b1d8ade178bb","Type":"ContainerDied","Data":"027f7a51458344f5bf563523c8bac523aa43701896513c6ad2176119037703f2"} May 21 11:33:36.679654 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:36.679621 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xf5gc" May 21 11:33:37.278829 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:37.278744 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-67f6f86555-zx427" event={"ID":"473c66ed-1f18-4de7-a9f3-34be157198ec","Type":"ContainerStarted","Data":"cd5d4fa1b76811419bb9bfd657042450f059150109113a3c010a9f79e05783b6"} May 21 11:33:37.278829 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:37.278786 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-67f6f86555-zx427" event={"ID":"473c66ed-1f18-4de7-a9f3-34be157198ec","Type":"ContainerStarted","Data":"214475e794b7fd3a8c1552b4aa5742d25f687a741bd1ed90cc8c3295643555fb"} May 21 11:33:37.278829 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:37.278798 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-67f6f86555-zx427" event={"ID":"473c66ed-1f18-4de7-a9f3-34be157198ec","Type":"ContainerStarted","Data":"f368f6e15cdbc0379d78fda018e3ea8be47c05e9c4fe6fbf337f0edcd52af077"} May 21 11:33:38.284474 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.284423 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-67f6f86555-zx427" event={"ID":"473c66ed-1f18-4de7-a9f3-34be157198ec","Type":"ContainerStarted","Data":"81b3a046581df5de6b6a966ddd47b1ac4d8d4c010061ba556e0f9e175f828c54"} May 21 11:33:38.286255 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.286231 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9ee6d1da-97d3-4f22-896c-b1d8ade178bb","Type":"ContainerStarted","Data":"7b04ee4b27e9020e0924a45aa2062a2fb954accee99e244c06775647f1a592b1"} May 21 11:33:38.588858 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.588829 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] May 21 11:33:38.592441 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.592412 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" May 21 11:33:38.598479 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.598311 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" May 21 11:33:38.598479 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.598337 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" May 21 11:33:38.598479 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.598482 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" May 21 11:33:38.598725 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.598622 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" May 21 11:33:38.598776 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.598742 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" May 21 11:33:38.599157 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.599009 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" May 21 11:33:38.599157 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.599044 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" May 21 11:33:38.599751 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.599728 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" May 21 11:33:38.599844 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.599750 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" May 21 11:33:38.601818 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.601626 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" May 21 11:33:38.601818 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.601716 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" May 21 11:33:38.601818 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.601779 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-49a9fvt3i4iv7\"" May 21 11:33:38.602511 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.602305 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" May 21 11:33:38.602849 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.602830 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-kpfmv\"" May 21 11:33:38.603909 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.603745 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" May 21 11:33:38.619536 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.619505 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] May 21 11:33:38.662313 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.662232 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ebbf5734-045d-4cea-bf46-f6ac80a11d46-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:33:38.662313 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.662285 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ebbf5734-045d-4cea-bf46-f6ac80a11d46-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:33:38.662313 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.662305 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ebbf5734-045d-4cea-bf46-f6ac80a11d46-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:33:38.662520 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.662322 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ebbf5734-045d-4cea-bf46-f6ac80a11d46-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:33:38.662520 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.662344 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ebbf5734-045d-4cea-bf46-f6ac80a11d46-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:33:38.662520 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.662362 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ebbf5734-045d-4cea-bf46-f6ac80a11d46-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:33:38.662520 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.662376 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9xbx\" (UniqueName: \"kubernetes.io/projected/ebbf5734-045d-4cea-bf46-f6ac80a11d46-kube-api-access-f9xbx\") pod \"prometheus-k8s-0\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:33:38.662520 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.662416 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ebbf5734-045d-4cea-bf46-f6ac80a11d46-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:33:38.662520 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.662488 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ebbf5734-045d-4cea-bf46-f6ac80a11d46-web-config\") pod \"prometheus-k8s-0\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:33:38.662686 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.662525 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ebbf5734-045d-4cea-bf46-f6ac80a11d46-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:33:38.662686 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.662548 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ebbf5734-045d-4cea-bf46-f6ac80a11d46-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:33:38.662686 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.662579 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ebbf5734-045d-4cea-bf46-f6ac80a11d46-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:33:38.662686 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.662605 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ebbf5734-045d-4cea-bf46-f6ac80a11d46-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:33:38.662686 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.662628 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ebbf5734-045d-4cea-bf46-f6ac80a11d46-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:33:38.662686 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.662647 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ebbf5734-045d-4cea-bf46-f6ac80a11d46-config\") pod \"prometheus-k8s-0\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:33:38.662686 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.662671 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ebbf5734-045d-4cea-bf46-f6ac80a11d46-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:33:38.662876 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.662714 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ebbf5734-045d-4cea-bf46-f6ac80a11d46-config-out\") pod \"prometheus-k8s-0\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:33:38.662876 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.662731 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ebbf5734-045d-4cea-bf46-f6ac80a11d46-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:33:38.763557 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.763510 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ebbf5734-045d-4cea-bf46-f6ac80a11d46-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:33:38.763557 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.763559 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ebbf5734-045d-4cea-bf46-f6ac80a11d46-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:33:38.763794 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.763579 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ebbf5734-045d-4cea-bf46-f6ac80a11d46-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:33:38.763794 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.763605 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ebbf5734-045d-4cea-bf46-f6ac80a11d46-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:33:38.763794 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.763631 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ebbf5734-045d-4cea-bf46-f6ac80a11d46-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:33:38.763794 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.763654 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-f9xbx\" (UniqueName: \"kubernetes.io/projected/ebbf5734-045d-4cea-bf46-f6ac80a11d46-kube-api-access-f9xbx\") pod \"prometheus-k8s-0\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:33:38.763794 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.763693 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ebbf5734-045d-4cea-bf46-f6ac80a11d46-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:33:38.764036 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.763855 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ebbf5734-045d-4cea-bf46-f6ac80a11d46-web-config\") pod \"prometheus-k8s-0\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:33:38.764036 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.763890 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ebbf5734-045d-4cea-bf46-f6ac80a11d46-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:33:38.764036 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.763917 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ebbf5734-045d-4cea-bf46-f6ac80a11d46-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:33:38.764036 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.763960 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ebbf5734-045d-4cea-bf46-f6ac80a11d46-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:33:38.764036 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.763994 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ebbf5734-045d-4cea-bf46-f6ac80a11d46-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:33:38.764036 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.764019 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ebbf5734-045d-4cea-bf46-f6ac80a11d46-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:33:38.764356 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.764055 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ebbf5734-045d-4cea-bf46-f6ac80a11d46-config\") pod \"prometheus-k8s-0\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:33:38.764356 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.764123 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ebbf5734-045d-4cea-bf46-f6ac80a11d46-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:33:38.764356 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.764158 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ebbf5734-045d-4cea-bf46-f6ac80a11d46-config-out\") pod \"prometheus-k8s-0\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:33:38.764356 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.764182 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ebbf5734-045d-4cea-bf46-f6ac80a11d46-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:33:38.764356 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.764238 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ebbf5734-045d-4cea-bf46-f6ac80a11d46-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:33:38.764656 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.764627 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ebbf5734-045d-4cea-bf46-f6ac80a11d46-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:33:38.765278 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.765246 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ebbf5734-045d-4cea-bf46-f6ac80a11d46-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:33:38.766152 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.765787 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ebbf5734-045d-4cea-bf46-f6ac80a11d46-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:33:38.766662 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.766635 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ebbf5734-045d-4cea-bf46-f6ac80a11d46-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:33:38.767320 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.767263 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ebbf5734-045d-4cea-bf46-f6ac80a11d46-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:33:38.767856 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.767498 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ebbf5734-045d-4cea-bf46-f6ac80a11d46-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:33:38.767856 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.767743 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ebbf5734-045d-4cea-bf46-f6ac80a11d46-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:33:38.768677 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.768652 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ebbf5734-045d-4cea-bf46-f6ac80a11d46-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:33:38.768825 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.768778 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ebbf5734-045d-4cea-bf46-f6ac80a11d46-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:33:38.769035 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.769014 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ebbf5734-045d-4cea-bf46-f6ac80a11d46-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:33:38.769276 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.769244 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ebbf5734-045d-4cea-bf46-f6ac80a11d46-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:33:38.769480 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.769459 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ebbf5734-045d-4cea-bf46-f6ac80a11d46-config-out\") pod \"prometheus-k8s-0\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:33:38.770049 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.770008 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ebbf5734-045d-4cea-bf46-f6ac80a11d46-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:33:38.770049 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.770032 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ebbf5734-045d-4cea-bf46-f6ac80a11d46-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:33:38.770592 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.770544 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ebbf5734-045d-4cea-bf46-f6ac80a11d46-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:33:38.771012 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.770988 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/ebbf5734-045d-4cea-bf46-f6ac80a11d46-config\") pod \"prometheus-k8s-0\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:33:38.771498 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.771458 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ebbf5734-045d-4cea-bf46-f6ac80a11d46-web-config\") pod \"prometheus-k8s-0\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:33:38.776669 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.776649 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9xbx\" (UniqueName: \"kubernetes.io/projected/ebbf5734-045d-4cea-bf46-f6ac80a11d46-kube-api-access-f9xbx\") pod \"prometheus-k8s-0\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:33:38.905605 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:38.905565 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" May 21 11:33:39.070121 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:39.066614 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] May 21 11:33:39.291888 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:39.291850 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-67f6f86555-zx427" event={"ID":"473c66ed-1f18-4de7-a9f3-34be157198ec","Type":"ContainerStarted","Data":"3a5722b798ae7a0049b62bfc5fb6ac7d156094d184ad45931b5802adef7ec911"} May 21 11:33:39.292375 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:39.291894 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-67f6f86555-zx427" event={"ID":"473c66ed-1f18-4de7-a9f3-34be157198ec","Type":"ContainerStarted","Data":"ed7eabecae501ed63fde5738e70e690a29bc8a04f200fd443952a42631f3a673"} May 21 11:33:39.292375 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:39.292104 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-67f6f86555-zx427" May 21 11:33:39.294995 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:39.294972 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9ee6d1da-97d3-4f22-896c-b1d8ade178bb","Type":"ContainerStarted","Data":"426ddc108ee276cc390278481116954ac15954d8eff3626be09187ff8287a804"} May 21 11:33:39.295139 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:39.295002 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9ee6d1da-97d3-4f22-896c-b1d8ade178bb","Type":"ContainerStarted","Data":"d2db9beef7060309112e64417a618adee7c9cac440fd5bbc2b7735c8377df85e"} May 21 11:33:39.295139 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:39.295015 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9ee6d1da-97d3-4f22-896c-b1d8ade178bb","Type":"ContainerStarted","Data":"a8acfcd931272820c030f4337318088a153b92d3219ebdfd6722f730c90da526"} May 21 11:33:39.295139 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:39.295024 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9ee6d1da-97d3-4f22-896c-b1d8ade178bb","Type":"ContainerStarted","Data":"7d96aa9ee31350ab188bfa0ab9914b570abded3343c6f752837d5a1717c98acd"} May 21 11:33:39.295139 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:39.295032 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9ee6d1da-97d3-4f22-896c-b1d8ade178bb","Type":"ContainerStarted","Data":"62f69ffd7e6c36d6ede94a15be245350c1ba474892bfa68c65e59d000717997b"} May 21 11:33:39.296453 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:39.296426 2564 generic.go:358] "Generic (PLEG): container finished" podID="ebbf5734-045d-4cea-bf46-f6ac80a11d46" containerID="a4754c31f4ee4b3378066adb6bb2ee88b912508f14405a39a46485225cd671a9" exitCode=0 May 21 11:33:39.296534 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:39.296481 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ebbf5734-045d-4cea-bf46-f6ac80a11d46","Type":"ContainerDied","Data":"a4754c31f4ee4b3378066adb6bb2ee88b912508f14405a39a46485225cd671a9"} May 21 11:33:39.296534 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:39.296506 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ebbf5734-045d-4cea-bf46-f6ac80a11d46","Type":"ContainerStarted","Data":"2042c2d856bb6b56629dae836fcde257cf1a61081bc443a6b08063ca3d5f2802"} May 21 11:33:39.316388 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:39.316343 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-67f6f86555-zx427" podStartSLOduration=2.171902053 podStartE2EDuration="5.316329937s" podCreationTimestamp="2026-05-21 11:33:34 +0000 UTC" firstStartedPulling="2026-05-21 11:33:34.677441329 +0000 UTC m=+169.572724853" lastFinishedPulling="2026-05-21 11:33:37.8218692 +0000 UTC m=+172.717152737" observedRunningTime="2026-05-21 11:33:39.314788665 +0000 UTC m=+174.210072211" watchObservedRunningTime="2026-05-21 11:33:39.316329937 +0000 UTC m=+174.211613532" May 21 11:33:39.348264 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:39.348213 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.6455876270000003 podStartE2EDuration="6.348199569s" podCreationTimestamp="2026-05-21 11:33:33 +0000 UTC" firstStartedPulling="2026-05-21 11:33:34.477049726 +0000 UTC m=+169.372333249" lastFinishedPulling="2026-05-21 11:33:38.179661661 +0000 UTC m=+173.074945191" observedRunningTime="2026-05-21 11:33:39.347609572 +0000 UTC m=+174.242893116" watchObservedRunningTime="2026-05-21 11:33:39.348199569 +0000 UTC m=+174.243483113" May 21 11:33:40.233992 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:40.233959 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-2pfk6" May 21 11:33:42.310299 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:42.310262 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ebbf5734-045d-4cea-bf46-f6ac80a11d46","Type":"ContainerStarted","Data":"d53fd38ac99a7365dd743fd4ab2c5e797a2569a3efdbdbf2e4473aae0f42cc37"} May 21 11:33:42.310647 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:42.310307 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ebbf5734-045d-4cea-bf46-f6ac80a11d46","Type":"ContainerStarted","Data":"39ad7f7fc43e2206d95a5f5a64dde7b712992b31b11b6f066d4368c28ab1830b"} May 21 11:33:42.310647 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:42.310320 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ebbf5734-045d-4cea-bf46-f6ac80a11d46","Type":"ContainerStarted","Data":"017288a0b2183b29a5f37805d020ce9e1837f78d53036a287bc90230386ef9c1"} May 21 11:33:42.310647 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:42.310332 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ebbf5734-045d-4cea-bf46-f6ac80a11d46","Type":"ContainerStarted","Data":"a7855dd2768f1c61da22a69cf17c8ed0149658247a862f542a1dab54a8888a96"} May 21 11:33:43.317060 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:43.317020 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ebbf5734-045d-4cea-bf46-f6ac80a11d46","Type":"ContainerStarted","Data":"0f6ca201cb9fadc254b1b71366129284226ac9a24eab2669a170548ba6296773"} May 21 11:33:43.317060 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:43.317058 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ebbf5734-045d-4cea-bf46-f6ac80a11d46","Type":"ContainerStarted","Data":"64a000f48b85757d642db3a1bdd29ffee6fbae528cbcade8d4c4487579183ca9"} May 21 11:33:43.351492 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:43.351424 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.681957181 podStartE2EDuration="5.351409789s" podCreationTimestamp="2026-05-21 11:33:38 +0000 UTC" firstStartedPulling="2026-05-21 11:33:39.29760033 +0000 UTC m=+174.192883854" lastFinishedPulling="2026-05-21 11:33:41.967052936 +0000 UTC m=+176.862336462" observedRunningTime="2026-05-21 11:33:43.350834389 +0000 UTC m=+178.246117934" watchObservedRunningTime="2026-05-21 11:33:43.351409789 +0000 UTC m=+178.246693503" May 21 11:33:43.684407 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:43.684332 2564 scope.go:117] "RemoveContainer" containerID="f06f3aafcc7dec3728eaad07da827eb505ef43d0d3542e47d5fd014562d0d7da" May 21 11:33:43.905968 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:43.905932 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" May 21 11:33:44.328399 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:44.328370 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-6g9xk_be463222-9e99-47cb-bf6b-525913ca9e29/console-operator/2.log" May 21 11:33:44.329007 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:44.328973 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-76dff48d59-6g9xk" event={"ID":"be463222-9e99-47cb-bf6b-525913ca9e29","Type":"ContainerStarted","Data":"84ba2d6461794b08aeafd43253af9b04e32da27826f150035d581319da8f6907"} May 21 11:33:44.330281 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:44.330249 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-76dff48d59-6g9xk" May 21 11:33:44.335135 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:44.335113 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-76dff48d59-6g9xk" May 21 11:33:44.351515 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:44.351470 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console-operator/console-operator-76dff48d59-6g9xk" podStartSLOduration=53.580940394 podStartE2EDuration="56.351458036s" podCreationTimestamp="2026-05-21 11:32:48 +0000 UTC" firstStartedPulling="2026-05-21 11:32:49.428673144 +0000 UTC m=+124.323956682" lastFinishedPulling="2026-05-21 11:32:52.199190785 +0000 UTC m=+127.094474324" observedRunningTime="2026-05-21 11:33:44.350458123 +0000 UTC m=+179.245741668" watchObservedRunningTime="2026-05-21 11:33:44.351458036 +0000 UTC m=+179.246741639" May 21 11:33:45.307373 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:45.307343 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-67f6f86555-zx427" May 21 11:33:58.375989 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:58.375955 2564 generic.go:358] "Generic (PLEG): container finished" podID="f797c9ec-815c-4e1a-aa83-9173927d8593" containerID="f47f02fae257a346d5a29829a184f682848eb8c5c03db5606ccffeeb24808489" exitCode=0 May 21 11:33:58.376398 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:58.376029 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-78bdfb888c-49stn" event={"ID":"f797c9ec-815c-4e1a-aa83-9173927d8593","Type":"ContainerDied","Data":"f47f02fae257a346d5a29829a184f682848eb8c5c03db5606ccffeeb24808489"} May 21 11:33:58.376398 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:58.376389 2564 scope.go:117] "RemoveContainer" containerID="f47f02fae257a346d5a29829a184f682848eb8c5c03db5606ccffeeb24808489" May 21 11:33:59.352600 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:59.352572 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-b7f67965d-272k2_4540e5a0-576b-4722-acfc-a2cec978f9e3/router/0.log" May 21 11:33:59.375705 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:59.375678 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-s79qs_b14cbac4-0b39-404a-a413-a5ccf5fb6472/serve-healthcheck-canary/0.log" May 21 11:33:59.380356 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:33:59.380333 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-78bdfb888c-49stn" event={"ID":"f797c9ec-815c-4e1a-aa83-9173927d8593","Type":"ContainerStarted","Data":"01eafc453b6e5b7cbd3e13e30e6d113dbad5e5546abf400e5aaaa97e6c32e538"} May 21 11:34:13.425531 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:13.425494 2564 generic.go:358] "Generic (PLEG): container finished" podID="a74a4dc1-ca52-40f4-9cb2-0c925a08ecfd" containerID="367fe8458136048878f86af42ecf2ad42f4311c74793ef8c785cb138661156b1" exitCode=0 May 21 11:34:13.425973 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:13.425572 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-xdnz5" event={"ID":"a74a4dc1-ca52-40f4-9cb2-0c925a08ecfd","Type":"ContainerDied","Data":"367fe8458136048878f86af42ecf2ad42f4311c74793ef8c785cb138661156b1"} May 21 11:34:13.425973 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:13.425911 2564 scope.go:117] "RemoveContainer" containerID="367fe8458136048878f86af42ecf2ad42f4311c74793ef8c785cb138661156b1" May 21 11:34:14.429934 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:14.429902 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-xdnz5" event={"ID":"a74a4dc1-ca52-40f4-9cb2-0c925a08ecfd","Type":"ContainerStarted","Data":"13165f9e3deb5494179f248e84ff9978be2039b9f409c700bde4d861878c4934"} May 21 11:34:38.906054 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:38.906017 2564 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:38.922511 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:38.922482 2564 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:39.518792 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:39.518762 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:52.573006 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:52.572969 2564 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] May 21 11:34:52.573499 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:52.573471 2564 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9ee6d1da-97d3-4f22-896c-b1d8ade178bb" containerName="alertmanager" containerID="cri-o://7b04ee4b27e9020e0924a45aa2062a2fb954accee99e244c06775647f1a592b1" gracePeriod=120 May 21 11:34:52.573589 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:52.573535 2564 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9ee6d1da-97d3-4f22-896c-b1d8ade178bb" containerName="kube-rbac-proxy-metric" containerID="cri-o://d2db9beef7060309112e64417a618adee7c9cac440fd5bbc2b7735c8377df85e" gracePeriod=120 May 21 11:34:52.573647 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:52.573602 2564 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9ee6d1da-97d3-4f22-896c-b1d8ade178bb" containerName="prom-label-proxy" containerID="cri-o://426ddc108ee276cc390278481116954ac15954d8eff3626be09187ff8287a804" gracePeriod=120 May 21 11:34:52.573705 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:52.573546 2564 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9ee6d1da-97d3-4f22-896c-b1d8ade178bb" containerName="kube-rbac-proxy-web" containerID="cri-o://7d96aa9ee31350ab188bfa0ab9914b570abded3343c6f752837d5a1717c98acd" gracePeriod=120 May 21 11:34:52.573705 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:52.573564 2564 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9ee6d1da-97d3-4f22-896c-b1d8ade178bb" containerName="config-reloader" containerID="cri-o://62f69ffd7e6c36d6ede94a15be245350c1ba474892bfa68c65e59d000717997b" gracePeriod=120 May 21 11:34:52.573705 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:52.573606 2564 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9ee6d1da-97d3-4f22-896c-b1d8ade178bb" containerName="kube-rbac-proxy" containerID="cri-o://a8acfcd931272820c030f4337318088a153b92d3219ebdfd6722f730c90da526" gracePeriod=120 May 21 11:34:53.546211 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:53.546177 2564 generic.go:358] "Generic (PLEG): container finished" podID="9ee6d1da-97d3-4f22-896c-b1d8ade178bb" containerID="426ddc108ee276cc390278481116954ac15954d8eff3626be09187ff8287a804" exitCode=0 May 21 11:34:53.546211 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:53.546204 2564 generic.go:358] "Generic (PLEG): container finished" podID="9ee6d1da-97d3-4f22-896c-b1d8ade178bb" containerID="a8acfcd931272820c030f4337318088a153b92d3219ebdfd6722f730c90da526" exitCode=0 May 21 11:34:53.546211 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:53.546212 2564 generic.go:358] "Generic (PLEG): container finished" podID="9ee6d1da-97d3-4f22-896c-b1d8ade178bb" containerID="62f69ffd7e6c36d6ede94a15be245350c1ba474892bfa68c65e59d000717997b" exitCode=0 May 21 11:34:53.546211 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:53.546219 2564 generic.go:358] "Generic (PLEG): container finished" podID="9ee6d1da-97d3-4f22-896c-b1d8ade178bb" containerID="7b04ee4b27e9020e0924a45aa2062a2fb954accee99e244c06775647f1a592b1" exitCode=0 May 21 11:34:53.546481 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:53.546249 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9ee6d1da-97d3-4f22-896c-b1d8ade178bb","Type":"ContainerDied","Data":"426ddc108ee276cc390278481116954ac15954d8eff3626be09187ff8287a804"} May 21 11:34:53.546481 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:53.546281 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9ee6d1da-97d3-4f22-896c-b1d8ade178bb","Type":"ContainerDied","Data":"a8acfcd931272820c030f4337318088a153b92d3219ebdfd6722f730c90da526"} May 21 11:34:53.546481 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:53.546291 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9ee6d1da-97d3-4f22-896c-b1d8ade178bb","Type":"ContainerDied","Data":"62f69ffd7e6c36d6ede94a15be245350c1ba474892bfa68c65e59d000717997b"} May 21 11:34:53.546481 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:53.546300 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9ee6d1da-97d3-4f22-896c-b1d8ade178bb","Type":"ContainerDied","Data":"7b04ee4b27e9020e0924a45aa2062a2fb954accee99e244c06775647f1a592b1"} May 21 11:34:53.830482 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:53.830459 2564 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" May 21 11:34:53.855020 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:53.854995 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-secret-alertmanager-kube-rbac-proxy-web\") pod \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\" (UID: \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\") " May 21 11:34:53.855169 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:53.855032 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-secret-alertmanager-kube-rbac-proxy\") pod \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\" (UID: \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\") " May 21 11:34:53.855169 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:53.855063 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-alertmanager-trusted-ca-bundle\") pod \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\" (UID: \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\") " May 21 11:34:53.855267 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:53.855207 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xm7kt\" (UniqueName: \"kubernetes.io/projected/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-kube-api-access-xm7kt\") pod \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\" (UID: \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\") " May 21 11:34:53.855267 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:53.855262 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-tls-assets\") pod \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\" (UID: \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\") " May 21 11:34:53.855367 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:53.855289 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-cluster-tls-config\") pod \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\" (UID: \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\") " May 21 11:34:53.855367 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:53.855312 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-metrics-client-ca\") pod \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\" (UID: \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\") " May 21 11:34:53.855367 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:53.855353 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-web-config\") pod \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\" (UID: \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\") " May 21 11:34:53.855517 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:53.855406 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-secret-alertmanager-kube-rbac-proxy-metric\") pod \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\" (UID: \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\") " May 21 11:34:53.855517 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:53.855430 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-config-volume\") pod \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\" (UID: \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\") " May 21 11:34:53.855517 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:53.855483 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-config-out\") pod \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\" (UID: \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\") " May 21 11:34:53.855517 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:53.855486 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "9ee6d1da-97d3-4f22-896c-b1d8ade178bb" (UID: "9ee6d1da-97d3-4f22-896c-b1d8ade178bb"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 21 11:34:53.855517 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:53.855516 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-secret-alertmanager-main-tls\") pod \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\" (UID: \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\") " May 21 11:34:53.855760 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:53.855550 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-alertmanager-main-db\") pod \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\" (UID: \"9ee6d1da-97d3-4f22-896c-b1d8ade178bb\") " May 21 11:34:53.855909 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:53.855751 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "9ee6d1da-97d3-4f22-896c-b1d8ade178bb" (UID: "9ee6d1da-97d3-4f22-896c-b1d8ade178bb"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 21 11:34:53.855909 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:53.855840 2564 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-139-213.ec2.internal\" DevicePath \"\"" May 21 11:34:53.855909 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:53.855860 2564 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-metrics-client-ca\") on node \"ip-10-0-139-213.ec2.internal\" DevicePath \"\"" May 21 11:34:53.856914 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:53.856881 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "9ee6d1da-97d3-4f22-896c-b1d8ade178bb" (UID: "9ee6d1da-97d3-4f22-896c-b1d8ade178bb"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 21 11:34:53.858320 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:53.858286 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-kube-api-access-xm7kt" (OuterVolumeSpecName: "kube-api-access-xm7kt") pod "9ee6d1da-97d3-4f22-896c-b1d8ade178bb" (UID: "9ee6d1da-97d3-4f22-896c-b1d8ade178bb"). InnerVolumeSpecName "kube-api-access-xm7kt". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 21 11:34:53.859440 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:53.859412 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "9ee6d1da-97d3-4f22-896c-b1d8ade178bb" (UID: "9ee6d1da-97d3-4f22-896c-b1d8ade178bb"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 21 11:34:53.859645 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:53.859547 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-config-out" (OuterVolumeSpecName: "config-out") pod "9ee6d1da-97d3-4f22-896c-b1d8ade178bb" (UID: "9ee6d1da-97d3-4f22-896c-b1d8ade178bb"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 21 11:34:53.859645 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:53.859602 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "9ee6d1da-97d3-4f22-896c-b1d8ade178bb" (UID: "9ee6d1da-97d3-4f22-896c-b1d8ade178bb"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 21 11:34:53.859645 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:53.859604 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "9ee6d1da-97d3-4f22-896c-b1d8ade178bb" (UID: "9ee6d1da-97d3-4f22-896c-b1d8ade178bb"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 21 11:34:53.859645 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:53.859621 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "9ee6d1da-97d3-4f22-896c-b1d8ade178bb" (UID: "9ee6d1da-97d3-4f22-896c-b1d8ade178bb"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 21 11:34:53.860331 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:53.860246 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-config-volume" (OuterVolumeSpecName: "config-volume") pod "9ee6d1da-97d3-4f22-896c-b1d8ade178bb" (UID: "9ee6d1da-97d3-4f22-896c-b1d8ade178bb"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 21 11:34:53.860331 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:53.860277 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "9ee6d1da-97d3-4f22-896c-b1d8ade178bb" (UID: "9ee6d1da-97d3-4f22-896c-b1d8ade178bb"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 21 11:34:53.864850 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:53.864796 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "9ee6d1da-97d3-4f22-896c-b1d8ade178bb" (UID: "9ee6d1da-97d3-4f22-896c-b1d8ade178bb"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 21 11:34:53.873810 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:53.873785 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-web-config" (OuterVolumeSpecName: "web-config") pod "9ee6d1da-97d3-4f22-896c-b1d8ade178bb" (UID: "9ee6d1da-97d3-4f22-896c-b1d8ade178bb"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 21 11:34:53.956866 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:53.956825 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xm7kt\" (UniqueName: \"kubernetes.io/projected/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-kube-api-access-xm7kt\") on node \"ip-10-0-139-213.ec2.internal\" DevicePath \"\"" May 21 11:34:53.956866 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:53.956855 2564 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-tls-assets\") on node \"ip-10-0-139-213.ec2.internal\" DevicePath \"\"" May 21 11:34:53.956866 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:53.956865 2564 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-cluster-tls-config\") on node \"ip-10-0-139-213.ec2.internal\" DevicePath \"\"" May 21 11:34:53.956866 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:53.956874 2564 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-web-config\") on node \"ip-10-0-139-213.ec2.internal\" DevicePath \"\"" May 21 11:34:53.957144 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:53.956884 2564 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-139-213.ec2.internal\" DevicePath \"\"" May 21 11:34:53.957144 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:53.956894 2564 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-config-volume\") on node \"ip-10-0-139-213.ec2.internal\" DevicePath \"\"" May 21 11:34:53.957144 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:53.956903 2564 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-config-out\") on node \"ip-10-0-139-213.ec2.internal\" DevicePath \"\"" May 21 11:34:53.957144 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:53.956911 2564 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-secret-alertmanager-main-tls\") on node \"ip-10-0-139-213.ec2.internal\" DevicePath \"\"" May 21 11:34:53.957144 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:53.956920 2564 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-alertmanager-main-db\") on node \"ip-10-0-139-213.ec2.internal\" DevicePath \"\"" May 21 11:34:53.957144 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:53.956929 2564 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-139-213.ec2.internal\" DevicePath \"\"" May 21 11:34:53.957144 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:53.956938 2564 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9ee6d1da-97d3-4f22-896c-b1d8ade178bb-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-139-213.ec2.internal\" DevicePath \"\"" May 21 11:34:54.551634 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.551601 2564 generic.go:358] "Generic (PLEG): container finished" podID="9ee6d1da-97d3-4f22-896c-b1d8ade178bb" containerID="d2db9beef7060309112e64417a618adee7c9cac440fd5bbc2b7735c8377df85e" exitCode=0 May 21 11:34:54.551634 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.551626 2564 generic.go:358] "Generic (PLEG): container finished" podID="9ee6d1da-97d3-4f22-896c-b1d8ade178bb" containerID="7d96aa9ee31350ab188bfa0ab9914b570abded3343c6f752837d5a1717c98acd" exitCode=0 May 21 11:34:54.551824 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.551669 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9ee6d1da-97d3-4f22-896c-b1d8ade178bb","Type":"ContainerDied","Data":"d2db9beef7060309112e64417a618adee7c9cac440fd5bbc2b7735c8377df85e"} May 21 11:34:54.551824 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.551710 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9ee6d1da-97d3-4f22-896c-b1d8ade178bb","Type":"ContainerDied","Data":"7d96aa9ee31350ab188bfa0ab9914b570abded3343c6f752837d5a1717c98acd"} May 21 11:34:54.551824 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.551723 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9ee6d1da-97d3-4f22-896c-b1d8ade178bb","Type":"ContainerDied","Data":"f46d14a8cac54049516fa3e2709b472821161c6857ae734c9839aeff3824f13b"} May 21 11:34:54.551824 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.551731 2564 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" May 21 11:34:54.551824 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.551737 2564 scope.go:117] "RemoveContainer" containerID="426ddc108ee276cc390278481116954ac15954d8eff3626be09187ff8287a804" May 21 11:34:54.560755 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.560739 2564 scope.go:117] "RemoveContainer" containerID="d2db9beef7060309112e64417a618adee7c9cac440fd5bbc2b7735c8377df85e" May 21 11:34:54.567704 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.567688 2564 scope.go:117] "RemoveContainer" containerID="a8acfcd931272820c030f4337318088a153b92d3219ebdfd6722f730c90da526" May 21 11:34:54.574256 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.574242 2564 scope.go:117] "RemoveContainer" containerID="7d96aa9ee31350ab188bfa0ab9914b570abded3343c6f752837d5a1717c98acd" May 21 11:34:54.579524 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.579504 2564 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] May 21 11:34:54.581484 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.581464 2564 scope.go:117] "RemoveContainer" containerID="62f69ffd7e6c36d6ede94a15be245350c1ba474892bfa68c65e59d000717997b" May 21 11:34:54.588277 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.588257 2564 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] May 21 11:34:54.588751 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.588740 2564 scope.go:117] "RemoveContainer" containerID="7b04ee4b27e9020e0924a45aa2062a2fb954accee99e244c06775647f1a592b1" May 21 11:34:54.595522 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.595507 2564 scope.go:117] "RemoveContainer" containerID="027f7a51458344f5bf563523c8bac523aa43701896513c6ad2176119037703f2" May 21 11:34:54.602070 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.602054 2564 scope.go:117] "RemoveContainer" containerID="426ddc108ee276cc390278481116954ac15954d8eff3626be09187ff8287a804" May 21 11:34:54.602345 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:34:54.602326 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"426ddc108ee276cc390278481116954ac15954d8eff3626be09187ff8287a804\": container with ID starting with 426ddc108ee276cc390278481116954ac15954d8eff3626be09187ff8287a804 not found: ID does not exist" containerID="426ddc108ee276cc390278481116954ac15954d8eff3626be09187ff8287a804" May 21 11:34:54.602412 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.602351 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"426ddc108ee276cc390278481116954ac15954d8eff3626be09187ff8287a804"} err="failed to get container status \"426ddc108ee276cc390278481116954ac15954d8eff3626be09187ff8287a804\": rpc error: code = NotFound desc = could not find container \"426ddc108ee276cc390278481116954ac15954d8eff3626be09187ff8287a804\": container with ID starting with 426ddc108ee276cc390278481116954ac15954d8eff3626be09187ff8287a804 not found: ID does not exist" May 21 11:34:54.602412 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.602389 2564 scope.go:117] "RemoveContainer" containerID="d2db9beef7060309112e64417a618adee7c9cac440fd5bbc2b7735c8377df85e" May 21 11:34:54.602613 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:34:54.602595 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2db9beef7060309112e64417a618adee7c9cac440fd5bbc2b7735c8377df85e\": container with ID starting with d2db9beef7060309112e64417a618adee7c9cac440fd5bbc2b7735c8377df85e not found: ID does not exist" containerID="d2db9beef7060309112e64417a618adee7c9cac440fd5bbc2b7735c8377df85e" May 21 11:34:54.602654 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.602620 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2db9beef7060309112e64417a618adee7c9cac440fd5bbc2b7735c8377df85e"} err="failed to get container status \"d2db9beef7060309112e64417a618adee7c9cac440fd5bbc2b7735c8377df85e\": rpc error: code = NotFound desc = could not find container \"d2db9beef7060309112e64417a618adee7c9cac440fd5bbc2b7735c8377df85e\": container with ID starting with d2db9beef7060309112e64417a618adee7c9cac440fd5bbc2b7735c8377df85e not found: ID does not exist" May 21 11:34:54.602654 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.602635 2564 scope.go:117] "RemoveContainer" containerID="a8acfcd931272820c030f4337318088a153b92d3219ebdfd6722f730c90da526" May 21 11:34:54.602874 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:34:54.602854 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8acfcd931272820c030f4337318088a153b92d3219ebdfd6722f730c90da526\": container with ID starting with a8acfcd931272820c030f4337318088a153b92d3219ebdfd6722f730c90da526 not found: ID does not exist" containerID="a8acfcd931272820c030f4337318088a153b92d3219ebdfd6722f730c90da526" May 21 11:34:54.602924 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.602881 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8acfcd931272820c030f4337318088a153b92d3219ebdfd6722f730c90da526"} err="failed to get container status \"a8acfcd931272820c030f4337318088a153b92d3219ebdfd6722f730c90da526\": rpc error: code = NotFound desc = could not find container \"a8acfcd931272820c030f4337318088a153b92d3219ebdfd6722f730c90da526\": container with ID starting with a8acfcd931272820c030f4337318088a153b92d3219ebdfd6722f730c90da526 not found: ID does not exist" May 21 11:34:54.602924 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.602899 2564 scope.go:117] "RemoveContainer" containerID="7d96aa9ee31350ab188bfa0ab9914b570abded3343c6f752837d5a1717c98acd" May 21 11:34:54.603132 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:34:54.603109 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d96aa9ee31350ab188bfa0ab9914b570abded3343c6f752837d5a1717c98acd\": container with ID starting with 7d96aa9ee31350ab188bfa0ab9914b570abded3343c6f752837d5a1717c98acd not found: ID does not exist" containerID="7d96aa9ee31350ab188bfa0ab9914b570abded3343c6f752837d5a1717c98acd" May 21 11:34:54.603224 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.603134 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d96aa9ee31350ab188bfa0ab9914b570abded3343c6f752837d5a1717c98acd"} err="failed to get container status \"7d96aa9ee31350ab188bfa0ab9914b570abded3343c6f752837d5a1717c98acd\": rpc error: code = NotFound desc = could not find container \"7d96aa9ee31350ab188bfa0ab9914b570abded3343c6f752837d5a1717c98acd\": container with ID starting with 7d96aa9ee31350ab188bfa0ab9914b570abded3343c6f752837d5a1717c98acd not found: ID does not exist" May 21 11:34:54.603224 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.603147 2564 scope.go:117] "RemoveContainer" containerID="62f69ffd7e6c36d6ede94a15be245350c1ba474892bfa68c65e59d000717997b" May 21 11:34:54.603408 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:34:54.603391 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62f69ffd7e6c36d6ede94a15be245350c1ba474892bfa68c65e59d000717997b\": container with ID starting with 62f69ffd7e6c36d6ede94a15be245350c1ba474892bfa68c65e59d000717997b not found: ID does not exist" containerID="62f69ffd7e6c36d6ede94a15be245350c1ba474892bfa68c65e59d000717997b" May 21 11:34:54.603475 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.603418 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62f69ffd7e6c36d6ede94a15be245350c1ba474892bfa68c65e59d000717997b"} err="failed to get container status \"62f69ffd7e6c36d6ede94a15be245350c1ba474892bfa68c65e59d000717997b\": rpc error: code = NotFound desc = could not find container \"62f69ffd7e6c36d6ede94a15be245350c1ba474892bfa68c65e59d000717997b\": container with ID starting with 62f69ffd7e6c36d6ede94a15be245350c1ba474892bfa68c65e59d000717997b not found: ID does not exist" May 21 11:34:54.603475 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.603438 2564 scope.go:117] "RemoveContainer" containerID="7b04ee4b27e9020e0924a45aa2062a2fb954accee99e244c06775647f1a592b1" May 21 11:34:54.603704 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:34:54.603688 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b04ee4b27e9020e0924a45aa2062a2fb954accee99e244c06775647f1a592b1\": container with ID starting with 7b04ee4b27e9020e0924a45aa2062a2fb954accee99e244c06775647f1a592b1 not found: ID does not exist" containerID="7b04ee4b27e9020e0924a45aa2062a2fb954accee99e244c06775647f1a592b1" May 21 11:34:54.603747 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.603709 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b04ee4b27e9020e0924a45aa2062a2fb954accee99e244c06775647f1a592b1"} err="failed to get container status \"7b04ee4b27e9020e0924a45aa2062a2fb954accee99e244c06775647f1a592b1\": rpc error: code = NotFound desc = could not find container \"7b04ee4b27e9020e0924a45aa2062a2fb954accee99e244c06775647f1a592b1\": container with ID starting with 7b04ee4b27e9020e0924a45aa2062a2fb954accee99e244c06775647f1a592b1 not found: ID does not exist" May 21 11:34:54.603747 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.603724 2564 scope.go:117] "RemoveContainer" containerID="027f7a51458344f5bf563523c8bac523aa43701896513c6ad2176119037703f2" May 21 11:34:54.603912 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:34:54.603898 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"027f7a51458344f5bf563523c8bac523aa43701896513c6ad2176119037703f2\": container with ID starting with 027f7a51458344f5bf563523c8bac523aa43701896513c6ad2176119037703f2 not found: ID does not exist" containerID="027f7a51458344f5bf563523c8bac523aa43701896513c6ad2176119037703f2" May 21 11:34:54.603952 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.603929 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"027f7a51458344f5bf563523c8bac523aa43701896513c6ad2176119037703f2"} err="failed to get container status \"027f7a51458344f5bf563523c8bac523aa43701896513c6ad2176119037703f2\": rpc error: code = NotFound desc = could not find container \"027f7a51458344f5bf563523c8bac523aa43701896513c6ad2176119037703f2\": container with ID starting with 027f7a51458344f5bf563523c8bac523aa43701896513c6ad2176119037703f2 not found: ID does not exist" May 21 11:34:54.603952 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.603941 2564 scope.go:117] "RemoveContainer" containerID="426ddc108ee276cc390278481116954ac15954d8eff3626be09187ff8287a804" May 21 11:34:54.604181 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.604162 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"426ddc108ee276cc390278481116954ac15954d8eff3626be09187ff8287a804"} err="failed to get container status \"426ddc108ee276cc390278481116954ac15954d8eff3626be09187ff8287a804\": rpc error: code = NotFound desc = could not find container \"426ddc108ee276cc390278481116954ac15954d8eff3626be09187ff8287a804\": container with ID starting with 426ddc108ee276cc390278481116954ac15954d8eff3626be09187ff8287a804 not found: ID does not exist" May 21 11:34:54.604237 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.604182 2564 scope.go:117] "RemoveContainer" containerID="d2db9beef7060309112e64417a618adee7c9cac440fd5bbc2b7735c8377df85e" May 21 11:34:54.604355 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.604340 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2db9beef7060309112e64417a618adee7c9cac440fd5bbc2b7735c8377df85e"} err="failed to get container status \"d2db9beef7060309112e64417a618adee7c9cac440fd5bbc2b7735c8377df85e\": rpc error: code = NotFound desc = could not find container \"d2db9beef7060309112e64417a618adee7c9cac440fd5bbc2b7735c8377df85e\": container with ID starting with d2db9beef7060309112e64417a618adee7c9cac440fd5bbc2b7735c8377df85e not found: ID does not exist" May 21 11:34:54.604402 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.604355 2564 scope.go:117] "RemoveContainer" containerID="a8acfcd931272820c030f4337318088a153b92d3219ebdfd6722f730c90da526" May 21 11:34:54.604563 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.604549 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8acfcd931272820c030f4337318088a153b92d3219ebdfd6722f730c90da526"} err="failed to get container status \"a8acfcd931272820c030f4337318088a153b92d3219ebdfd6722f730c90da526\": rpc error: code = NotFound desc = could not find container \"a8acfcd931272820c030f4337318088a153b92d3219ebdfd6722f730c90da526\": container with ID starting with a8acfcd931272820c030f4337318088a153b92d3219ebdfd6722f730c90da526 not found: ID does not exist" May 21 11:34:54.604602 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.604565 2564 scope.go:117] "RemoveContainer" containerID="7d96aa9ee31350ab188bfa0ab9914b570abded3343c6f752837d5a1717c98acd" May 21 11:34:54.604768 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.604752 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d96aa9ee31350ab188bfa0ab9914b570abded3343c6f752837d5a1717c98acd"} err="failed to get container status \"7d96aa9ee31350ab188bfa0ab9914b570abded3343c6f752837d5a1717c98acd\": rpc error: code = NotFound desc = could not find container \"7d96aa9ee31350ab188bfa0ab9914b570abded3343c6f752837d5a1717c98acd\": container with ID starting with 7d96aa9ee31350ab188bfa0ab9914b570abded3343c6f752837d5a1717c98acd not found: ID does not exist" May 21 11:34:54.604810 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.604769 2564 scope.go:117] "RemoveContainer" containerID="62f69ffd7e6c36d6ede94a15be245350c1ba474892bfa68c65e59d000717997b" May 21 11:34:54.604957 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.604937 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62f69ffd7e6c36d6ede94a15be245350c1ba474892bfa68c65e59d000717997b"} err="failed to get container status \"62f69ffd7e6c36d6ede94a15be245350c1ba474892bfa68c65e59d000717997b\": rpc error: code = NotFound desc = could not find container \"62f69ffd7e6c36d6ede94a15be245350c1ba474892bfa68c65e59d000717997b\": container with ID starting with 62f69ffd7e6c36d6ede94a15be245350c1ba474892bfa68c65e59d000717997b not found: ID does not exist" May 21 11:34:54.605005 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.604957 2564 scope.go:117] "RemoveContainer" containerID="7b04ee4b27e9020e0924a45aa2062a2fb954accee99e244c06775647f1a592b1" May 21 11:34:54.605147 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.605132 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b04ee4b27e9020e0924a45aa2062a2fb954accee99e244c06775647f1a592b1"} err="failed to get container status \"7b04ee4b27e9020e0924a45aa2062a2fb954accee99e244c06775647f1a592b1\": rpc error: code = NotFound desc = could not find container \"7b04ee4b27e9020e0924a45aa2062a2fb954accee99e244c06775647f1a592b1\": container with ID starting with 7b04ee4b27e9020e0924a45aa2062a2fb954accee99e244c06775647f1a592b1 not found: ID does not exist" May 21 11:34:54.605197 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.605147 2564 scope.go:117] "RemoveContainer" containerID="027f7a51458344f5bf563523c8bac523aa43701896513c6ad2176119037703f2" May 21 11:34:54.607716 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.605541 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"027f7a51458344f5bf563523c8bac523aa43701896513c6ad2176119037703f2"} err="failed to get container status \"027f7a51458344f5bf563523c8bac523aa43701896513c6ad2176119037703f2\": rpc error: code = NotFound desc = could not find container \"027f7a51458344f5bf563523c8bac523aa43701896513c6ad2176119037703f2\": container with ID starting with 027f7a51458344f5bf563523c8bac523aa43701896513c6ad2176119037703f2 not found: ID does not exist" May 21 11:34:54.619588 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.619565 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] May 21 11:34:54.619914 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.619901 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ee6d1da-97d3-4f22-896c-b1d8ade178bb" containerName="init-config-reloader" May 21 11:34:54.619966 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.619916 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ee6d1da-97d3-4f22-896c-b1d8ade178bb" containerName="init-config-reloader" May 21 11:34:54.619966 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.619926 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ee6d1da-97d3-4f22-896c-b1d8ade178bb" containerName="kube-rbac-proxy-web" May 21 11:34:54.619966 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.619931 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ee6d1da-97d3-4f22-896c-b1d8ade178bb" containerName="kube-rbac-proxy-web" May 21 11:34:54.619966 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.619942 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ee6d1da-97d3-4f22-896c-b1d8ade178bb" containerName="alertmanager" May 21 11:34:54.619966 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.619947 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ee6d1da-97d3-4f22-896c-b1d8ade178bb" containerName="alertmanager" May 21 11:34:54.619966 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.619956 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ee6d1da-97d3-4f22-896c-b1d8ade178bb" containerName="config-reloader" May 21 11:34:54.619966 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.619961 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ee6d1da-97d3-4f22-896c-b1d8ade178bb" containerName="config-reloader" May 21 11:34:54.620207 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.619970 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ee6d1da-97d3-4f22-896c-b1d8ade178bb" containerName="kube-rbac-proxy" May 21 11:34:54.620207 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.619975 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ee6d1da-97d3-4f22-896c-b1d8ade178bb" containerName="kube-rbac-proxy" May 21 11:34:54.620207 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.619985 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ee6d1da-97d3-4f22-896c-b1d8ade178bb" containerName="kube-rbac-proxy-metric" May 21 11:34:54.620207 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.619990 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ee6d1da-97d3-4f22-896c-b1d8ade178bb" containerName="kube-rbac-proxy-metric" May 21 11:34:54.620207 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.620000 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ee6d1da-97d3-4f22-896c-b1d8ade178bb" containerName="prom-label-proxy" May 21 11:34:54.620207 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.620006 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ee6d1da-97d3-4f22-896c-b1d8ade178bb" containerName="prom-label-proxy" May 21 11:34:54.620207 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.620053 2564 memory_manager.go:356] "RemoveStaleState removing state" podUID="9ee6d1da-97d3-4f22-896c-b1d8ade178bb" containerName="kube-rbac-proxy-metric" May 21 11:34:54.620207 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.620062 2564 memory_manager.go:356] "RemoveStaleState removing state" podUID="9ee6d1da-97d3-4f22-896c-b1d8ade178bb" containerName="prom-label-proxy" May 21 11:34:54.620207 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.620069 2564 memory_manager.go:356] "RemoveStaleState removing state" podUID="9ee6d1da-97d3-4f22-896c-b1d8ade178bb" containerName="config-reloader" May 21 11:34:54.620207 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.620094 2564 memory_manager.go:356] "RemoveStaleState removing state" podUID="9ee6d1da-97d3-4f22-896c-b1d8ade178bb" containerName="kube-rbac-proxy" May 21 11:34:54.620207 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.620105 2564 memory_manager.go:356] "RemoveStaleState removing state" podUID="9ee6d1da-97d3-4f22-896c-b1d8ade178bb" containerName="alertmanager" May 21 11:34:54.620207 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.620112 2564 memory_manager.go:356] "RemoveStaleState removing state" podUID="9ee6d1da-97d3-4f22-896c-b1d8ade178bb" containerName="kube-rbac-proxy-web" May 21 11:34:54.624023 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.624007 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" May 21 11:34:54.627124 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.627106 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" May 21 11:34:54.627214 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.627109 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" May 21 11:34:54.627214 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.627110 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" May 21 11:34:54.627607 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.627562 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" May 21 11:34:54.627704 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.627606 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" May 21 11:34:54.627704 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.627627 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" May 21 11:34:54.627909 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.627895 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" May 21 11:34:54.627978 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.627912 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-kpmvs\"" May 21 11:34:54.628113 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.628067 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" May 21 11:34:54.634100 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.634064 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" May 21 11:34:54.638312 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.638288 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] May 21 11:34:54.662855 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.662824 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3ed3c3ab-07ec-404a-9f28-41e03eabaeaa-config-out\") pod \"alertmanager-main-0\" (UID: \"3ed3c3ab-07ec-404a-9f28-41e03eabaeaa\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:34:54.662976 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.662869 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3ed3c3ab-07ec-404a-9f28-41e03eabaeaa-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3ed3c3ab-07ec-404a-9f28-41e03eabaeaa\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:34:54.662976 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.662925 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3ed3c3ab-07ec-404a-9f28-41e03eabaeaa-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3ed3c3ab-07ec-404a-9f28-41e03eabaeaa\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:34:54.662976 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.662960 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3ed3c3ab-07ec-404a-9f28-41e03eabaeaa-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3ed3c3ab-07ec-404a-9f28-41e03eabaeaa\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:34:54.663160 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.662988 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3ed3c3ab-07ec-404a-9f28-41e03eabaeaa-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3ed3c3ab-07ec-404a-9f28-41e03eabaeaa\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:34:54.663160 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.663033 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ed3c3ab-07ec-404a-9f28-41e03eabaeaa-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3ed3c3ab-07ec-404a-9f28-41e03eabaeaa\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:34:54.663160 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.663065 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3ed3c3ab-07ec-404a-9f28-41e03eabaeaa-config-volume\") pod \"alertmanager-main-0\" (UID: \"3ed3c3ab-07ec-404a-9f28-41e03eabaeaa\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:34:54.663271 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.663172 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3ed3c3ab-07ec-404a-9f28-41e03eabaeaa-web-config\") pod \"alertmanager-main-0\" (UID: \"3ed3c3ab-07ec-404a-9f28-41e03eabaeaa\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:34:54.663271 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.663202 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ed3c3ab-07ec-404a-9f28-41e03eabaeaa-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3ed3c3ab-07ec-404a-9f28-41e03eabaeaa\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:34:54.663364 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.663339 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdp4v\" (UniqueName: \"kubernetes.io/projected/3ed3c3ab-07ec-404a-9f28-41e03eabaeaa-kube-api-access-cdp4v\") pod \"alertmanager-main-0\" (UID: \"3ed3c3ab-07ec-404a-9f28-41e03eabaeaa\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:34:54.663398 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.663376 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3ed3c3ab-07ec-404a-9f28-41e03eabaeaa-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3ed3c3ab-07ec-404a-9f28-41e03eabaeaa\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:34:54.663430 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.663398 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3ed3c3ab-07ec-404a-9f28-41e03eabaeaa-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3ed3c3ab-07ec-404a-9f28-41e03eabaeaa\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:34:54.663494 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.663477 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3ed3c3ab-07ec-404a-9f28-41e03eabaeaa-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3ed3c3ab-07ec-404a-9f28-41e03eabaeaa\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:34:54.763936 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.763906 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3ed3c3ab-07ec-404a-9f28-41e03eabaeaa-config-out\") pod \"alertmanager-main-0\" (UID: \"3ed3c3ab-07ec-404a-9f28-41e03eabaeaa\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:34:54.764108 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.763940 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3ed3c3ab-07ec-404a-9f28-41e03eabaeaa-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3ed3c3ab-07ec-404a-9f28-41e03eabaeaa\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:34:54.764108 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.763961 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3ed3c3ab-07ec-404a-9f28-41e03eabaeaa-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3ed3c3ab-07ec-404a-9f28-41e03eabaeaa\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:34:54.764108 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.763982 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3ed3c3ab-07ec-404a-9f28-41e03eabaeaa-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3ed3c3ab-07ec-404a-9f28-41e03eabaeaa\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:34:54.764108 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.763999 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3ed3c3ab-07ec-404a-9f28-41e03eabaeaa-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3ed3c3ab-07ec-404a-9f28-41e03eabaeaa\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:34:54.764108 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.764032 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ed3c3ab-07ec-404a-9f28-41e03eabaeaa-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3ed3c3ab-07ec-404a-9f28-41e03eabaeaa\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:34:54.764108 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.764049 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3ed3c3ab-07ec-404a-9f28-41e03eabaeaa-config-volume\") pod \"alertmanager-main-0\" (UID: \"3ed3c3ab-07ec-404a-9f28-41e03eabaeaa\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:34:54.764108 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.764072 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3ed3c3ab-07ec-404a-9f28-41e03eabaeaa-web-config\") pod \"alertmanager-main-0\" (UID: \"3ed3c3ab-07ec-404a-9f28-41e03eabaeaa\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:34:54.764451 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.764132 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ed3c3ab-07ec-404a-9f28-41e03eabaeaa-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3ed3c3ab-07ec-404a-9f28-41e03eabaeaa\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:34:54.764451 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.764220 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cdp4v\" (UniqueName: \"kubernetes.io/projected/3ed3c3ab-07ec-404a-9f28-41e03eabaeaa-kube-api-access-cdp4v\") pod \"alertmanager-main-0\" (UID: \"3ed3c3ab-07ec-404a-9f28-41e03eabaeaa\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:34:54.764451 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.764249 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3ed3c3ab-07ec-404a-9f28-41e03eabaeaa-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3ed3c3ab-07ec-404a-9f28-41e03eabaeaa\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:34:54.764451 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.764278 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3ed3c3ab-07ec-404a-9f28-41e03eabaeaa-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3ed3c3ab-07ec-404a-9f28-41e03eabaeaa\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:34:54.764451 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.764306 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3ed3c3ab-07ec-404a-9f28-41e03eabaeaa-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3ed3c3ab-07ec-404a-9f28-41e03eabaeaa\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:34:54.764685 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.764467 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3ed3c3ab-07ec-404a-9f28-41e03eabaeaa-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3ed3c3ab-07ec-404a-9f28-41e03eabaeaa\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:34:54.765395 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.765177 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ed3c3ab-07ec-404a-9f28-41e03eabaeaa-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3ed3c3ab-07ec-404a-9f28-41e03eabaeaa\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:34:54.767726 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.767064 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3ed3c3ab-07ec-404a-9f28-41e03eabaeaa-config-out\") pod \"alertmanager-main-0\" (UID: \"3ed3c3ab-07ec-404a-9f28-41e03eabaeaa\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:34:54.767726 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.767336 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3ed3c3ab-07ec-404a-9f28-41e03eabaeaa-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3ed3c3ab-07ec-404a-9f28-41e03eabaeaa\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:34:54.767726 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.767438 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3ed3c3ab-07ec-404a-9f28-41e03eabaeaa-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3ed3c3ab-07ec-404a-9f28-41e03eabaeaa\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:34:54.767726 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.767571 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3ed3c3ab-07ec-404a-9f28-41e03eabaeaa-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3ed3c3ab-07ec-404a-9f28-41e03eabaeaa\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:34:54.767726 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.767649 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3ed3c3ab-07ec-404a-9f28-41e03eabaeaa-web-config\") pod \"alertmanager-main-0\" (UID: \"3ed3c3ab-07ec-404a-9f28-41e03eabaeaa\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:34:54.767726 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.767687 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ed3c3ab-07ec-404a-9f28-41e03eabaeaa-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3ed3c3ab-07ec-404a-9f28-41e03eabaeaa\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:34:54.768039 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.767850 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3ed3c3ab-07ec-404a-9f28-41e03eabaeaa-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3ed3c3ab-07ec-404a-9f28-41e03eabaeaa\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:34:54.768039 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.767851 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3ed3c3ab-07ec-404a-9f28-41e03eabaeaa-config-volume\") pod \"alertmanager-main-0\" (UID: \"3ed3c3ab-07ec-404a-9f28-41e03eabaeaa\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:34:54.768039 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.767978 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3ed3c3ab-07ec-404a-9f28-41e03eabaeaa-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3ed3c3ab-07ec-404a-9f28-41e03eabaeaa\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:34:54.769361 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.769341 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3ed3c3ab-07ec-404a-9f28-41e03eabaeaa-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3ed3c3ab-07ec-404a-9f28-41e03eabaeaa\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:34:54.773703 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.773682 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdp4v\" (UniqueName: \"kubernetes.io/projected/3ed3c3ab-07ec-404a-9f28-41e03eabaeaa-kube-api-access-cdp4v\") pod \"alertmanager-main-0\" (UID: \"3ed3c3ab-07ec-404a-9f28-41e03eabaeaa\") " pod="openshift-monitoring/alertmanager-main-0" May 21 11:34:54.934561 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:54.934479 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" May 21 11:34:55.087401 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:34:55.087372 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ed3c3ab_07ec_404a_9f28_41e03eabaeaa.slice/crio-e0a9abf89765aee9ca8a7aa71f859b78330ae01106480d844143f131a8b3da1a WatchSource:0}: Error finding container e0a9abf89765aee9ca8a7aa71f859b78330ae01106480d844143f131a8b3da1a: Status 404 returned error can't find the container with id e0a9abf89765aee9ca8a7aa71f859b78330ae01106480d844143f131a8b3da1a May 21 11:34:55.091541 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:55.091515 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] May 21 11:34:55.556614 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:55.556581 2564 generic.go:358] "Generic (PLEG): container finished" podID="3ed3c3ab-07ec-404a-9f28-41e03eabaeaa" containerID="68882adfbdf1ce83de550510833e25c4b0d5b2d192f0e174ecabef1722f59341" exitCode=0 May 21 11:34:55.556791 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:55.556648 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3ed3c3ab-07ec-404a-9f28-41e03eabaeaa","Type":"ContainerDied","Data":"68882adfbdf1ce83de550510833e25c4b0d5b2d192f0e174ecabef1722f59341"} May 21 11:34:55.556791 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:55.556668 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3ed3c3ab-07ec-404a-9f28-41e03eabaeaa","Type":"ContainerStarted","Data":"e0a9abf89765aee9ca8a7aa71f859b78330ae01106480d844143f131a8b3da1a"} May 21 11:34:55.688593 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:55.688564 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ee6d1da-97d3-4f22-896c-b1d8ade178bb" path="/var/lib/kubelet/pods/9ee6d1da-97d3-4f22-896c-b1d8ade178bb/volumes" May 21 11:34:56.563813 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:56.563771 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3ed3c3ab-07ec-404a-9f28-41e03eabaeaa","Type":"ContainerStarted","Data":"89acb252fc549e0e5f7528957c63166aab14dc6cd0b491c353a8dd80716b792e"} May 21 11:34:56.564322 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:56.563819 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3ed3c3ab-07ec-404a-9f28-41e03eabaeaa","Type":"ContainerStarted","Data":"8f695ea281ceab1bf6d096f0a96f8e43dce04cd63fefecede9074937b67eb231"} May 21 11:34:56.564322 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:56.563836 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3ed3c3ab-07ec-404a-9f28-41e03eabaeaa","Type":"ContainerStarted","Data":"80da89492b8e63b1d837d2ee83bd4d2dd779f43fb33104fbc6ba030408bf8d66"} May 21 11:34:56.564322 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:56.563849 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3ed3c3ab-07ec-404a-9f28-41e03eabaeaa","Type":"ContainerStarted","Data":"112388ff4a9642de22e9a6b257c60be7ae167c31d05ca794a326ea0431c867a8"} May 21 11:34:56.564322 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:56.563860 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3ed3c3ab-07ec-404a-9f28-41e03eabaeaa","Type":"ContainerStarted","Data":"7086c1741183b1ceb5e4397220c1ebe020d0fc331dac189c7db8a22fd0a81531"} May 21 11:34:56.564322 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:56.563872 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3ed3c3ab-07ec-404a-9f28-41e03eabaeaa","Type":"ContainerStarted","Data":"67e95761e92191d97676819708f282ae2cea4556a799110cf1be75966cc11033"} May 21 11:34:56.583100 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:56.583045 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/414e12fe-b9ac-4bbf-8470-ad442215f5cb-metrics-certs\") pod \"network-metrics-daemon-xf5gc\" (UID: \"414e12fe-b9ac-4bbf-8470-ad442215f5cb\") " pod="openshift-multus/network-metrics-daemon-xf5gc" May 21 11:34:56.585667 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:56.585635 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/414e12fe-b9ac-4bbf-8470-ad442215f5cb-metrics-certs\") pod \"network-metrics-daemon-xf5gc\" (UID: \"414e12fe-b9ac-4bbf-8470-ad442215f5cb\") " pod="openshift-multus/network-metrics-daemon-xf5gc" May 21 11:34:56.783675 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:56.783637 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tr4cg\"" May 21 11:34:56.791230 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:56.791198 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xf5gc" May 21 11:34:56.908802 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:56.908730 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.908707069 podStartE2EDuration="2.908707069s" podCreationTimestamp="2026-05-21 11:34:54 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-21 11:34:56.602043184 +0000 UTC m=+251.497326728" watchObservedRunningTime="2026-05-21 11:34:56.908707069 +0000 UTC m=+251.803990616" May 21 11:34:56.910345 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:56.910319 2564 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] May 21 11:34:56.910997 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:56.910765 2564 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ebbf5734-045d-4cea-bf46-f6ac80a11d46" containerName="kube-rbac-proxy-web" containerID="cri-o://d53fd38ac99a7365dd743fd4ab2c5e797a2569a3efdbdbf2e4473aae0f42cc37" gracePeriod=600 May 21 11:34:56.910997 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:56.910765 2564 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ebbf5734-045d-4cea-bf46-f6ac80a11d46" containerName="kube-rbac-proxy" containerID="cri-o://64a000f48b85757d642db3a1bdd29ffee6fbae528cbcade8d4c4487579183ca9" gracePeriod=600 May 21 11:34:56.910997 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:56.910809 2564 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ebbf5734-045d-4cea-bf46-f6ac80a11d46" containerName="kube-rbac-proxy-thanos" containerID="cri-o://0f6ca201cb9fadc254b1b71366129284226ac9a24eab2669a170548ba6296773" gracePeriod=600 May 21 11:34:56.910997 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:56.910778 2564 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ebbf5734-045d-4cea-bf46-f6ac80a11d46" containerName="thanos-sidecar" containerID="cri-o://39ad7f7fc43e2206d95a5f5a64dde7b712992b31b11b6f066d4368c28ab1830b" gracePeriod=600 May 21 11:34:56.910997 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:56.910844 2564 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ebbf5734-045d-4cea-bf46-f6ac80a11d46" containerName="config-reloader" containerID="cri-o://017288a0b2183b29a5f37805d020ce9e1837f78d53036a287bc90230386ef9c1" gracePeriod=600 May 21 11:34:56.910997 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:56.910898 2564 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ebbf5734-045d-4cea-bf46-f6ac80a11d46" containerName="prometheus" containerID="cri-o://a7855dd2768f1c61da22a69cf17c8ed0149658247a862f542a1dab54a8888a96" gracePeriod=600 May 21 11:34:56.934234 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:56.934203 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xf5gc"] May 21 11:34:56.938725 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:34:56.938684 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod414e12fe_b9ac_4bbf_8470_ad442215f5cb.slice/crio-a956b776a995b56b71592a2d69ed2ea8ecf78bc92067b4b861fbfd0b75326064 WatchSource:0}: Error finding container a956b776a995b56b71592a2d69ed2ea8ecf78bc92067b4b861fbfd0b75326064: Status 404 returned error can't find the container with id a956b776a995b56b71592a2d69ed2ea8ecf78bc92067b4b861fbfd0b75326064 May 21 11:34:57.166574 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.166547 2564 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:57.289768 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.289734 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ebbf5734-045d-4cea-bf46-f6ac80a11d46-configmap-serving-certs-ca-bundle\") pod \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " May 21 11:34:57.289930 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.289778 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ebbf5734-045d-4cea-bf46-f6ac80a11d46-prometheus-trusted-ca-bundle\") pod \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " May 21 11:34:57.289930 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.289798 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ebbf5734-045d-4cea-bf46-f6ac80a11d46-secret-metrics-client-certs\") pod \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " May 21 11:34:57.289930 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.289823 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ebbf5734-045d-4cea-bf46-f6ac80a11d46-web-config\") pod \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " May 21 11:34:57.289930 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.289841 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ebbf5734-045d-4cea-bf46-f6ac80a11d46-configmap-metrics-client-ca\") pod \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " May 21 11:34:57.289930 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.289861 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ebbf5734-045d-4cea-bf46-f6ac80a11d46-prometheus-k8s-db\") pod \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " May 21 11:34:57.289930 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.289894 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ebbf5734-045d-4cea-bf46-f6ac80a11d46-tls-assets\") pod \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " May 21 11:34:57.290262 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.289933 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ebbf5734-045d-4cea-bf46-f6ac80a11d46-config-out\") pod \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " May 21 11:34:57.290262 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.289962 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ebbf5734-045d-4cea-bf46-f6ac80a11d46-configmap-kubelet-serving-ca-bundle\") pod \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " May 21 11:34:57.290262 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.289994 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ebbf5734-045d-4cea-bf46-f6ac80a11d46-secret-kube-rbac-proxy\") pod \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " May 21 11:34:57.290262 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.290020 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ebbf5734-045d-4cea-bf46-f6ac80a11d46-secret-grpc-tls\") pod \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " May 21 11:34:57.290262 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.290036 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ebbf5734-045d-4cea-bf46-f6ac80a11d46-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " May 21 11:34:57.290262 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.290106 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ebbf5734-045d-4cea-bf46-f6ac80a11d46-secret-prometheus-k8s-tls\") pod \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " May 21 11:34:57.290262 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.290132 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ebbf5734-045d-4cea-bf46-f6ac80a11d46-thanos-prometheus-http-client-file\") pod \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " May 21 11:34:57.290262 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.290152 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ebbf5734-045d-4cea-bf46-f6ac80a11d46-prometheus-k8s-rulefiles-0\") pod \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " May 21 11:34:57.290262 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.290169 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ebbf5734-045d-4cea-bf46-f6ac80a11d46-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " May 21 11:34:57.290262 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.290204 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ebbf5734-045d-4cea-bf46-f6ac80a11d46-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "ebbf5734-045d-4cea-bf46-f6ac80a11d46" (UID: "ebbf5734-045d-4cea-bf46-f6ac80a11d46"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 21 11:34:57.290262 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.290211 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ebbf5734-045d-4cea-bf46-f6ac80a11d46-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "ebbf5734-045d-4cea-bf46-f6ac80a11d46" (UID: "ebbf5734-045d-4cea-bf46-f6ac80a11d46"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 21 11:34:57.290262 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.290217 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f9xbx\" (UniqueName: \"kubernetes.io/projected/ebbf5734-045d-4cea-bf46-f6ac80a11d46-kube-api-access-f9xbx\") pod \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " May 21 11:34:57.290835 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.290283 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ebbf5734-045d-4cea-bf46-f6ac80a11d46-config\") pod \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\" (UID: \"ebbf5734-045d-4cea-bf46-f6ac80a11d46\") " May 21 11:34:57.290835 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.290326 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ebbf5734-045d-4cea-bf46-f6ac80a11d46-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "ebbf5734-045d-4cea-bf46-f6ac80a11d46" (UID: "ebbf5734-045d-4cea-bf46-f6ac80a11d46"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 21 11:34:57.290835 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.290474 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ebbf5734-045d-4cea-bf46-f6ac80a11d46-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "ebbf5734-045d-4cea-bf46-f6ac80a11d46" (UID: "ebbf5734-045d-4cea-bf46-f6ac80a11d46"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 21 11:34:57.290835 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.290781 2564 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ebbf5734-045d-4cea-bf46-f6ac80a11d46-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-139-213.ec2.internal\" DevicePath \"\"" May 21 11:34:57.290835 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.290819 2564 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ebbf5734-045d-4cea-bf46-f6ac80a11d46-prometheus-trusted-ca-bundle\") on node \"ip-10-0-139-213.ec2.internal\" DevicePath \"\"" May 21 11:34:57.290835 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.290836 2564 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ebbf5734-045d-4cea-bf46-f6ac80a11d46-configmap-metrics-client-ca\") on node \"ip-10-0-139-213.ec2.internal\" DevicePath \"\"" May 21 11:34:57.291222 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.290851 2564 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ebbf5734-045d-4cea-bf46-f6ac80a11d46-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-139-213.ec2.internal\" DevicePath \"\"" May 21 11:34:57.291222 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.291190 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebbf5734-045d-4cea-bf46-f6ac80a11d46-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "ebbf5734-045d-4cea-bf46-f6ac80a11d46" (UID: "ebbf5734-045d-4cea-bf46-f6ac80a11d46"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 21 11:34:57.292051 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.292014 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ebbf5734-045d-4cea-bf46-f6ac80a11d46-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "ebbf5734-045d-4cea-bf46-f6ac80a11d46" (UID: "ebbf5734-045d-4cea-bf46-f6ac80a11d46"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 21 11:34:57.293285 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.293258 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ebbf5734-045d-4cea-bf46-f6ac80a11d46-config" (OuterVolumeSpecName: "config") pod "ebbf5734-045d-4cea-bf46-f6ac80a11d46" (UID: "ebbf5734-045d-4cea-bf46-f6ac80a11d46"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 21 11:34:57.293873 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.293831 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebbf5734-045d-4cea-bf46-f6ac80a11d46-kube-api-access-f9xbx" (OuterVolumeSpecName: "kube-api-access-f9xbx") pod "ebbf5734-045d-4cea-bf46-f6ac80a11d46" (UID: "ebbf5734-045d-4cea-bf46-f6ac80a11d46"). InnerVolumeSpecName "kube-api-access-f9xbx". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 21 11:34:57.293873 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.293838 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ebbf5734-045d-4cea-bf46-f6ac80a11d46-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "ebbf5734-045d-4cea-bf46-f6ac80a11d46" (UID: "ebbf5734-045d-4cea-bf46-f6ac80a11d46"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 21 11:34:57.294524 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.294491 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ebbf5734-045d-4cea-bf46-f6ac80a11d46-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "ebbf5734-045d-4cea-bf46-f6ac80a11d46" (UID: "ebbf5734-045d-4cea-bf46-f6ac80a11d46"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 21 11:34:57.294620 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.294557 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ebbf5734-045d-4cea-bf46-f6ac80a11d46-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "ebbf5734-045d-4cea-bf46-f6ac80a11d46" (UID: "ebbf5734-045d-4cea-bf46-f6ac80a11d46"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 21 11:34:57.295123 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.295069 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ebbf5734-045d-4cea-bf46-f6ac80a11d46-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "ebbf5734-045d-4cea-bf46-f6ac80a11d46" (UID: "ebbf5734-045d-4cea-bf46-f6ac80a11d46"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 21 11:34:57.295221 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.295164 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ebbf5734-045d-4cea-bf46-f6ac80a11d46-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "ebbf5734-045d-4cea-bf46-f6ac80a11d46" (UID: "ebbf5734-045d-4cea-bf46-f6ac80a11d46"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 21 11:34:57.295325 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.295300 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebbf5734-045d-4cea-bf46-f6ac80a11d46-config-out" (OuterVolumeSpecName: "config-out") pod "ebbf5734-045d-4cea-bf46-f6ac80a11d46" (UID: "ebbf5734-045d-4cea-bf46-f6ac80a11d46"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 21 11:34:57.295484 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.295460 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ebbf5734-045d-4cea-bf46-f6ac80a11d46-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "ebbf5734-045d-4cea-bf46-f6ac80a11d46" (UID: "ebbf5734-045d-4cea-bf46-f6ac80a11d46"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 21 11:34:57.295567 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.295521 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebbf5734-045d-4cea-bf46-f6ac80a11d46-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "ebbf5734-045d-4cea-bf46-f6ac80a11d46" (UID: "ebbf5734-045d-4cea-bf46-f6ac80a11d46"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 21 11:34:57.295613 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.295590 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ebbf5734-045d-4cea-bf46-f6ac80a11d46-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "ebbf5734-045d-4cea-bf46-f6ac80a11d46" (UID: "ebbf5734-045d-4cea-bf46-f6ac80a11d46"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 21 11:34:57.305103 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.305063 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ebbf5734-045d-4cea-bf46-f6ac80a11d46-web-config" (OuterVolumeSpecName: "web-config") pod "ebbf5734-045d-4cea-bf46-f6ac80a11d46" (UID: "ebbf5734-045d-4cea-bf46-f6ac80a11d46"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 21 11:34:57.391859 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.391779 2564 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ebbf5734-045d-4cea-bf46-f6ac80a11d46-web-config\") on node \"ip-10-0-139-213.ec2.internal\" DevicePath \"\"" May 21 11:34:57.391859 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.391805 2564 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ebbf5734-045d-4cea-bf46-f6ac80a11d46-prometheus-k8s-db\") on node \"ip-10-0-139-213.ec2.internal\" DevicePath \"\"" May 21 11:34:57.391859 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.391815 2564 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ebbf5734-045d-4cea-bf46-f6ac80a11d46-tls-assets\") on node \"ip-10-0-139-213.ec2.internal\" DevicePath \"\"" May 21 11:34:57.391859 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.391823 2564 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ebbf5734-045d-4cea-bf46-f6ac80a11d46-config-out\") on node \"ip-10-0-139-213.ec2.internal\" DevicePath \"\"" May 21 11:34:57.391859 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.391834 2564 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ebbf5734-045d-4cea-bf46-f6ac80a11d46-secret-kube-rbac-proxy\") on node \"ip-10-0-139-213.ec2.internal\" DevicePath \"\"" May 21 11:34:57.391859 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.391859 2564 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ebbf5734-045d-4cea-bf46-f6ac80a11d46-secret-grpc-tls\") on node \"ip-10-0-139-213.ec2.internal\" DevicePath \"\"" May 21 11:34:57.392169 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.391868 2564 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ebbf5734-045d-4cea-bf46-f6ac80a11d46-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-139-213.ec2.internal\" DevicePath \"\"" May 21 11:34:57.392169 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.391877 2564 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ebbf5734-045d-4cea-bf46-f6ac80a11d46-secret-prometheus-k8s-tls\") on node \"ip-10-0-139-213.ec2.internal\" DevicePath \"\"" May 21 11:34:57.392169 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.391886 2564 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ebbf5734-045d-4cea-bf46-f6ac80a11d46-thanos-prometheus-http-client-file\") on node \"ip-10-0-139-213.ec2.internal\" DevicePath \"\"" May 21 11:34:57.392169 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.391895 2564 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ebbf5734-045d-4cea-bf46-f6ac80a11d46-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-139-213.ec2.internal\" DevicePath \"\"" May 21 11:34:57.392169 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.391903 2564 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ebbf5734-045d-4cea-bf46-f6ac80a11d46-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-139-213.ec2.internal\" DevicePath \"\"" May 21 11:34:57.392169 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.391912 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f9xbx\" (UniqueName: \"kubernetes.io/projected/ebbf5734-045d-4cea-bf46-f6ac80a11d46-kube-api-access-f9xbx\") on node \"ip-10-0-139-213.ec2.internal\" DevicePath \"\"" May 21 11:34:57.392169 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.391920 2564 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/ebbf5734-045d-4cea-bf46-f6ac80a11d46-config\") on node \"ip-10-0-139-213.ec2.internal\" DevicePath \"\"" May 21 11:34:57.392169 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.391928 2564 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ebbf5734-045d-4cea-bf46-f6ac80a11d46-secret-metrics-client-certs\") on node \"ip-10-0-139-213.ec2.internal\" DevicePath \"\"" May 21 11:34:57.571387 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.571349 2564 generic.go:358] "Generic (PLEG): container finished" podID="ebbf5734-045d-4cea-bf46-f6ac80a11d46" containerID="0f6ca201cb9fadc254b1b71366129284226ac9a24eab2669a170548ba6296773" exitCode=0 May 21 11:34:57.571387 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.571382 2564 generic.go:358] "Generic (PLEG): container finished" podID="ebbf5734-045d-4cea-bf46-f6ac80a11d46" containerID="64a000f48b85757d642db3a1bdd29ffee6fbae528cbcade8d4c4487579183ca9" exitCode=0 May 21 11:34:57.571387 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.571391 2564 generic.go:358] "Generic (PLEG): container finished" podID="ebbf5734-045d-4cea-bf46-f6ac80a11d46" containerID="d53fd38ac99a7365dd743fd4ab2c5e797a2569a3efdbdbf2e4473aae0f42cc37" exitCode=0 May 21 11:34:57.571949 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.571400 2564 generic.go:358] "Generic (PLEG): container finished" podID="ebbf5734-045d-4cea-bf46-f6ac80a11d46" containerID="39ad7f7fc43e2206d95a5f5a64dde7b712992b31b11b6f066d4368c28ab1830b" exitCode=0 May 21 11:34:57.571949 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.571408 2564 generic.go:358] "Generic (PLEG): container finished" podID="ebbf5734-045d-4cea-bf46-f6ac80a11d46" containerID="017288a0b2183b29a5f37805d020ce9e1837f78d53036a287bc90230386ef9c1" exitCode=0 May 21 11:34:57.571949 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.571415 2564 generic.go:358] "Generic (PLEG): container finished" podID="ebbf5734-045d-4cea-bf46-f6ac80a11d46" containerID="a7855dd2768f1c61da22a69cf17c8ed0149658247a862f542a1dab54a8888a96" exitCode=0 May 21 11:34:57.571949 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.571445 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ebbf5734-045d-4cea-bf46-f6ac80a11d46","Type":"ContainerDied","Data":"0f6ca201cb9fadc254b1b71366129284226ac9a24eab2669a170548ba6296773"} May 21 11:34:57.571949 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.571501 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ebbf5734-045d-4cea-bf46-f6ac80a11d46","Type":"ContainerDied","Data":"64a000f48b85757d642db3a1bdd29ffee6fbae528cbcade8d4c4487579183ca9"} May 21 11:34:57.571949 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.571520 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ebbf5734-045d-4cea-bf46-f6ac80a11d46","Type":"ContainerDied","Data":"d53fd38ac99a7365dd743fd4ab2c5e797a2569a3efdbdbf2e4473aae0f42cc37"} May 21 11:34:57.571949 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.571503 2564 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:57.571949 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.571530 2564 scope.go:117] "RemoveContainer" containerID="0f6ca201cb9fadc254b1b71366129284226ac9a24eab2669a170548ba6296773" May 21 11:34:57.571949 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.571537 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ebbf5734-045d-4cea-bf46-f6ac80a11d46","Type":"ContainerDied","Data":"39ad7f7fc43e2206d95a5f5a64dde7b712992b31b11b6f066d4368c28ab1830b"} May 21 11:34:57.571949 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.571553 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ebbf5734-045d-4cea-bf46-f6ac80a11d46","Type":"ContainerDied","Data":"017288a0b2183b29a5f37805d020ce9e1837f78d53036a287bc90230386ef9c1"} May 21 11:34:57.571949 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.571565 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ebbf5734-045d-4cea-bf46-f6ac80a11d46","Type":"ContainerDied","Data":"a7855dd2768f1c61da22a69cf17c8ed0149658247a862f542a1dab54a8888a96"} May 21 11:34:57.571949 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.571578 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ebbf5734-045d-4cea-bf46-f6ac80a11d46","Type":"ContainerDied","Data":"2042c2d856bb6b56629dae836fcde257cf1a61081bc443a6b08063ca3d5f2802"} May 21 11:34:57.573352 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.573299 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xf5gc" event={"ID":"414e12fe-b9ac-4bbf-8470-ad442215f5cb","Type":"ContainerStarted","Data":"a956b776a995b56b71592a2d69ed2ea8ecf78bc92067b4b861fbfd0b75326064"} May 21 11:34:57.583571 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.583417 2564 scope.go:117] "RemoveContainer" containerID="64a000f48b85757d642db3a1bdd29ffee6fbae528cbcade8d4c4487579183ca9" May 21 11:34:57.592860 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.592836 2564 scope.go:117] "RemoveContainer" containerID="d53fd38ac99a7365dd743fd4ab2c5e797a2569a3efdbdbf2e4473aae0f42cc37" May 21 11:34:57.602543 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.602519 2564 scope.go:117] "RemoveContainer" containerID="39ad7f7fc43e2206d95a5f5a64dde7b712992b31b11b6f066d4368c28ab1830b" May 21 11:34:57.603103 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.603054 2564 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] May 21 11:34:57.607247 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.607215 2564 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] May 21 11:34:57.611861 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.611840 2564 scope.go:117] "RemoveContainer" containerID="017288a0b2183b29a5f37805d020ce9e1837f78d53036a287bc90230386ef9c1" May 21 11:34:57.620902 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.620875 2564 scope.go:117] "RemoveContainer" containerID="a7855dd2768f1c61da22a69cf17c8ed0149658247a862f542a1dab54a8888a96" May 21 11:34:57.630910 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.630887 2564 scope.go:117] "RemoveContainer" containerID="a4754c31f4ee4b3378066adb6bb2ee88b912508f14405a39a46485225cd671a9" May 21 11:34:57.631638 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.631614 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] May 21 11:34:57.632111 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.632094 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebbf5734-045d-4cea-bf46-f6ac80a11d46" containerName="kube-rbac-proxy-thanos" May 21 11:34:57.632234 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.632115 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebbf5734-045d-4cea-bf46-f6ac80a11d46" containerName="kube-rbac-proxy-thanos" May 21 11:34:57.632234 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.632139 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebbf5734-045d-4cea-bf46-f6ac80a11d46" containerName="thanos-sidecar" May 21 11:34:57.632234 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.632147 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebbf5734-045d-4cea-bf46-f6ac80a11d46" containerName="thanos-sidecar" May 21 11:34:57.632234 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.632158 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebbf5734-045d-4cea-bf46-f6ac80a11d46" containerName="kube-rbac-proxy" May 21 11:34:57.632234 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.632167 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebbf5734-045d-4cea-bf46-f6ac80a11d46" containerName="kube-rbac-proxy" May 21 11:34:57.632234 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.632180 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebbf5734-045d-4cea-bf46-f6ac80a11d46" containerName="init-config-reloader" May 21 11:34:57.632234 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.632188 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebbf5734-045d-4cea-bf46-f6ac80a11d46" containerName="init-config-reloader" May 21 11:34:57.632234 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.632202 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebbf5734-045d-4cea-bf46-f6ac80a11d46" containerName="kube-rbac-proxy-web" May 21 11:34:57.632234 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.632211 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebbf5734-045d-4cea-bf46-f6ac80a11d46" containerName="kube-rbac-proxy-web" May 21 11:34:57.632234 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.632232 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebbf5734-045d-4cea-bf46-f6ac80a11d46" containerName="prometheus" May 21 11:34:57.632234 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.632240 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebbf5734-045d-4cea-bf46-f6ac80a11d46" containerName="prometheus" May 21 11:34:57.632597 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.632248 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebbf5734-045d-4cea-bf46-f6ac80a11d46" containerName="config-reloader" May 21 11:34:57.632597 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.632253 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebbf5734-045d-4cea-bf46-f6ac80a11d46" containerName="config-reloader" May 21 11:34:57.632597 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.632328 2564 memory_manager.go:356] "RemoveStaleState removing state" podUID="ebbf5734-045d-4cea-bf46-f6ac80a11d46" containerName="config-reloader" May 21 11:34:57.632597 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.632337 2564 memory_manager.go:356] "RemoveStaleState removing state" podUID="ebbf5734-045d-4cea-bf46-f6ac80a11d46" containerName="kube-rbac-proxy" May 21 11:34:57.632597 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.632346 2564 memory_manager.go:356] "RemoveStaleState removing state" podUID="ebbf5734-045d-4cea-bf46-f6ac80a11d46" containerName="prometheus" May 21 11:34:57.632597 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.632352 2564 memory_manager.go:356] "RemoveStaleState removing state" podUID="ebbf5734-045d-4cea-bf46-f6ac80a11d46" containerName="thanos-sidecar" May 21 11:34:57.632597 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.632359 2564 memory_manager.go:356] "RemoveStaleState removing state" podUID="ebbf5734-045d-4cea-bf46-f6ac80a11d46" containerName="kube-rbac-proxy-thanos" May 21 11:34:57.632597 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.632365 2564 memory_manager.go:356] "RemoveStaleState removing state" podUID="ebbf5734-045d-4cea-bf46-f6ac80a11d46" containerName="kube-rbac-proxy-web" May 21 11:34:57.637708 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.637686 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:57.641227 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.641026 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" May 21 11:34:57.641227 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.641033 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" May 21 11:34:57.641480 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.641392 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-kpfmv\"" May 21 11:34:57.641733 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.641564 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-49a9fvt3i4iv7\"" May 21 11:34:57.641733 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.641584 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" May 21 11:34:57.641805 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.641765 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" May 21 11:34:57.642067 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.642010 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" May 21 11:34:57.642281 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.642238 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" May 21 11:34:57.643544 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.642713 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" May 21 11:34:57.643544 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.642821 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" May 21 11:34:57.643544 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.642938 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" May 21 11:34:57.643544 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.643148 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" May 21 11:34:57.643544 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.643163 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" May 21 11:34:57.644859 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.644814 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" May 21 11:34:57.655760 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.655601 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" May 21 11:34:57.656717 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.656692 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] May 21 11:34:57.662528 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.662503 2564 scope.go:117] "RemoveContainer" containerID="0f6ca201cb9fadc254b1b71366129284226ac9a24eab2669a170548ba6296773" May 21 11:34:57.663051 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:34:57.663023 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f6ca201cb9fadc254b1b71366129284226ac9a24eab2669a170548ba6296773\": container with ID starting with 0f6ca201cb9fadc254b1b71366129284226ac9a24eab2669a170548ba6296773 not found: ID does not exist" containerID="0f6ca201cb9fadc254b1b71366129284226ac9a24eab2669a170548ba6296773" May 21 11:34:57.663160 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.663061 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f6ca201cb9fadc254b1b71366129284226ac9a24eab2669a170548ba6296773"} err="failed to get container status \"0f6ca201cb9fadc254b1b71366129284226ac9a24eab2669a170548ba6296773\": rpc error: code = NotFound desc = could not find container \"0f6ca201cb9fadc254b1b71366129284226ac9a24eab2669a170548ba6296773\": container with ID starting with 0f6ca201cb9fadc254b1b71366129284226ac9a24eab2669a170548ba6296773 not found: ID does not exist" May 21 11:34:57.663160 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.663118 2564 scope.go:117] "RemoveContainer" containerID="64a000f48b85757d642db3a1bdd29ffee6fbae528cbcade8d4c4487579183ca9" May 21 11:34:57.663439 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:34:57.663410 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64a000f48b85757d642db3a1bdd29ffee6fbae528cbcade8d4c4487579183ca9\": container with ID starting with 64a000f48b85757d642db3a1bdd29ffee6fbae528cbcade8d4c4487579183ca9 not found: ID does not exist" containerID="64a000f48b85757d642db3a1bdd29ffee6fbae528cbcade8d4c4487579183ca9" May 21 11:34:57.663512 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.663447 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64a000f48b85757d642db3a1bdd29ffee6fbae528cbcade8d4c4487579183ca9"} err="failed to get container status \"64a000f48b85757d642db3a1bdd29ffee6fbae528cbcade8d4c4487579183ca9\": rpc error: code = NotFound desc = could not find container \"64a000f48b85757d642db3a1bdd29ffee6fbae528cbcade8d4c4487579183ca9\": container with ID starting with 64a000f48b85757d642db3a1bdd29ffee6fbae528cbcade8d4c4487579183ca9 not found: ID does not exist" May 21 11:34:57.663512 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.663470 2564 scope.go:117] "RemoveContainer" containerID="d53fd38ac99a7365dd743fd4ab2c5e797a2569a3efdbdbf2e4473aae0f42cc37" May 21 11:34:57.663764 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:34:57.663743 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d53fd38ac99a7365dd743fd4ab2c5e797a2569a3efdbdbf2e4473aae0f42cc37\": container with ID starting with d53fd38ac99a7365dd743fd4ab2c5e797a2569a3efdbdbf2e4473aae0f42cc37 not found: ID does not exist" containerID="d53fd38ac99a7365dd743fd4ab2c5e797a2569a3efdbdbf2e4473aae0f42cc37" May 21 11:34:57.663845 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.663770 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d53fd38ac99a7365dd743fd4ab2c5e797a2569a3efdbdbf2e4473aae0f42cc37"} err="failed to get container status \"d53fd38ac99a7365dd743fd4ab2c5e797a2569a3efdbdbf2e4473aae0f42cc37\": rpc error: code = NotFound desc = could not find container \"d53fd38ac99a7365dd743fd4ab2c5e797a2569a3efdbdbf2e4473aae0f42cc37\": container with ID starting with d53fd38ac99a7365dd743fd4ab2c5e797a2569a3efdbdbf2e4473aae0f42cc37 not found: ID does not exist" May 21 11:34:57.663845 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.663789 2564 scope.go:117] "RemoveContainer" containerID="39ad7f7fc43e2206d95a5f5a64dde7b712992b31b11b6f066d4368c28ab1830b" May 21 11:34:57.664064 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:34:57.664047 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39ad7f7fc43e2206d95a5f5a64dde7b712992b31b11b6f066d4368c28ab1830b\": container with ID starting with 39ad7f7fc43e2206d95a5f5a64dde7b712992b31b11b6f066d4368c28ab1830b not found: ID does not exist" containerID="39ad7f7fc43e2206d95a5f5a64dde7b712992b31b11b6f066d4368c28ab1830b" May 21 11:34:57.664170 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.664064 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39ad7f7fc43e2206d95a5f5a64dde7b712992b31b11b6f066d4368c28ab1830b"} err="failed to get container status \"39ad7f7fc43e2206d95a5f5a64dde7b712992b31b11b6f066d4368c28ab1830b\": rpc error: code = NotFound desc = could not find container \"39ad7f7fc43e2206d95a5f5a64dde7b712992b31b11b6f066d4368c28ab1830b\": container with ID starting with 39ad7f7fc43e2206d95a5f5a64dde7b712992b31b11b6f066d4368c28ab1830b not found: ID does not exist" May 21 11:34:57.664170 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.664103 2564 scope.go:117] "RemoveContainer" containerID="017288a0b2183b29a5f37805d020ce9e1837f78d53036a287bc90230386ef9c1" May 21 11:34:57.664508 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:34:57.664479 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"017288a0b2183b29a5f37805d020ce9e1837f78d53036a287bc90230386ef9c1\": container with ID starting with 017288a0b2183b29a5f37805d020ce9e1837f78d53036a287bc90230386ef9c1 not found: ID does not exist" containerID="017288a0b2183b29a5f37805d020ce9e1837f78d53036a287bc90230386ef9c1" May 21 11:34:57.664604 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.664515 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"017288a0b2183b29a5f37805d020ce9e1837f78d53036a287bc90230386ef9c1"} err="failed to get container status \"017288a0b2183b29a5f37805d020ce9e1837f78d53036a287bc90230386ef9c1\": rpc error: code = NotFound desc = could not find container \"017288a0b2183b29a5f37805d020ce9e1837f78d53036a287bc90230386ef9c1\": container with ID starting with 017288a0b2183b29a5f37805d020ce9e1837f78d53036a287bc90230386ef9c1 not found: ID does not exist" May 21 11:34:57.664604 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.664579 2564 scope.go:117] "RemoveContainer" containerID="a7855dd2768f1c61da22a69cf17c8ed0149658247a862f542a1dab54a8888a96" May 21 11:34:57.664929 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:34:57.664906 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7855dd2768f1c61da22a69cf17c8ed0149658247a862f542a1dab54a8888a96\": container with ID starting with a7855dd2768f1c61da22a69cf17c8ed0149658247a862f542a1dab54a8888a96 not found: ID does not exist" containerID="a7855dd2768f1c61da22a69cf17c8ed0149658247a862f542a1dab54a8888a96" May 21 11:34:57.665004 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.664936 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7855dd2768f1c61da22a69cf17c8ed0149658247a862f542a1dab54a8888a96"} err="failed to get container status \"a7855dd2768f1c61da22a69cf17c8ed0149658247a862f542a1dab54a8888a96\": rpc error: code = NotFound desc = could not find container \"a7855dd2768f1c61da22a69cf17c8ed0149658247a862f542a1dab54a8888a96\": container with ID starting with a7855dd2768f1c61da22a69cf17c8ed0149658247a862f542a1dab54a8888a96 not found: ID does not exist" May 21 11:34:57.665004 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.664957 2564 scope.go:117] "RemoveContainer" containerID="a4754c31f4ee4b3378066adb6bb2ee88b912508f14405a39a46485225cd671a9" May 21 11:34:57.665357 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:34:57.665259 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4754c31f4ee4b3378066adb6bb2ee88b912508f14405a39a46485225cd671a9\": container with ID starting with a4754c31f4ee4b3378066adb6bb2ee88b912508f14405a39a46485225cd671a9 not found: ID does not exist" containerID="a4754c31f4ee4b3378066adb6bb2ee88b912508f14405a39a46485225cd671a9" May 21 11:34:57.665357 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.665287 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4754c31f4ee4b3378066adb6bb2ee88b912508f14405a39a46485225cd671a9"} err="failed to get container status \"a4754c31f4ee4b3378066adb6bb2ee88b912508f14405a39a46485225cd671a9\": rpc error: code = NotFound desc = could not find container \"a4754c31f4ee4b3378066adb6bb2ee88b912508f14405a39a46485225cd671a9\": container with ID starting with a4754c31f4ee4b3378066adb6bb2ee88b912508f14405a39a46485225cd671a9 not found: ID does not exist" May 21 11:34:57.665357 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.665307 2564 scope.go:117] "RemoveContainer" containerID="0f6ca201cb9fadc254b1b71366129284226ac9a24eab2669a170548ba6296773" May 21 11:34:57.665532 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.665511 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f6ca201cb9fadc254b1b71366129284226ac9a24eab2669a170548ba6296773"} err="failed to get container status \"0f6ca201cb9fadc254b1b71366129284226ac9a24eab2669a170548ba6296773\": rpc error: code = NotFound desc = could not find container \"0f6ca201cb9fadc254b1b71366129284226ac9a24eab2669a170548ba6296773\": container with ID starting with 0f6ca201cb9fadc254b1b71366129284226ac9a24eab2669a170548ba6296773 not found: ID does not exist" May 21 11:34:57.665654 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.665640 2564 scope.go:117] "RemoveContainer" containerID="64a000f48b85757d642db3a1bdd29ffee6fbae528cbcade8d4c4487579183ca9" May 21 11:34:57.667489 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.667461 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64a000f48b85757d642db3a1bdd29ffee6fbae528cbcade8d4c4487579183ca9"} err="failed to get container status \"64a000f48b85757d642db3a1bdd29ffee6fbae528cbcade8d4c4487579183ca9\": rpc error: code = NotFound desc = could not find container \"64a000f48b85757d642db3a1bdd29ffee6fbae528cbcade8d4c4487579183ca9\": container with ID starting with 64a000f48b85757d642db3a1bdd29ffee6fbae528cbcade8d4c4487579183ca9 not found: ID does not exist" May 21 11:34:57.667489 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.667490 2564 scope.go:117] "RemoveContainer" containerID="d53fd38ac99a7365dd743fd4ab2c5e797a2569a3efdbdbf2e4473aae0f42cc37" May 21 11:34:57.667818 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.667793 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d53fd38ac99a7365dd743fd4ab2c5e797a2569a3efdbdbf2e4473aae0f42cc37"} err="failed to get container status \"d53fd38ac99a7365dd743fd4ab2c5e797a2569a3efdbdbf2e4473aae0f42cc37\": rpc error: code = NotFound desc = could not find container \"d53fd38ac99a7365dd743fd4ab2c5e797a2569a3efdbdbf2e4473aae0f42cc37\": container with ID starting with d53fd38ac99a7365dd743fd4ab2c5e797a2569a3efdbdbf2e4473aae0f42cc37 not found: ID does not exist" May 21 11:34:57.668019 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.668002 2564 scope.go:117] "RemoveContainer" containerID="39ad7f7fc43e2206d95a5f5a64dde7b712992b31b11b6f066d4368c28ab1830b" May 21 11:34:57.668492 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.668465 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39ad7f7fc43e2206d95a5f5a64dde7b712992b31b11b6f066d4368c28ab1830b"} err="failed to get container status \"39ad7f7fc43e2206d95a5f5a64dde7b712992b31b11b6f066d4368c28ab1830b\": rpc error: code = NotFound desc = could not find container \"39ad7f7fc43e2206d95a5f5a64dde7b712992b31b11b6f066d4368c28ab1830b\": container with ID starting with 39ad7f7fc43e2206d95a5f5a64dde7b712992b31b11b6f066d4368c28ab1830b not found: ID does not exist" May 21 11:34:57.668492 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.668493 2564 scope.go:117] "RemoveContainer" containerID="017288a0b2183b29a5f37805d020ce9e1837f78d53036a287bc90230386ef9c1" May 21 11:34:57.668776 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.668752 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"017288a0b2183b29a5f37805d020ce9e1837f78d53036a287bc90230386ef9c1"} err="failed to get container status \"017288a0b2183b29a5f37805d020ce9e1837f78d53036a287bc90230386ef9c1\": rpc error: code = NotFound desc = could not find container \"017288a0b2183b29a5f37805d020ce9e1837f78d53036a287bc90230386ef9c1\": container with ID starting with 017288a0b2183b29a5f37805d020ce9e1837f78d53036a287bc90230386ef9c1 not found: ID does not exist" May 21 11:34:57.668875 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.668779 2564 scope.go:117] "RemoveContainer" containerID="a7855dd2768f1c61da22a69cf17c8ed0149658247a862f542a1dab54a8888a96" May 21 11:34:57.669057 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.669039 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7855dd2768f1c61da22a69cf17c8ed0149658247a862f542a1dab54a8888a96"} err="failed to get container status \"a7855dd2768f1c61da22a69cf17c8ed0149658247a862f542a1dab54a8888a96\": rpc error: code = NotFound desc = could not find container \"a7855dd2768f1c61da22a69cf17c8ed0149658247a862f542a1dab54a8888a96\": container with ID starting with a7855dd2768f1c61da22a69cf17c8ed0149658247a862f542a1dab54a8888a96 not found: ID does not exist" May 21 11:34:57.669178 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.669059 2564 scope.go:117] "RemoveContainer" containerID="a4754c31f4ee4b3378066adb6bb2ee88b912508f14405a39a46485225cd671a9" May 21 11:34:57.669359 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.669336 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4754c31f4ee4b3378066adb6bb2ee88b912508f14405a39a46485225cd671a9"} err="failed to get container status \"a4754c31f4ee4b3378066adb6bb2ee88b912508f14405a39a46485225cd671a9\": rpc error: code = NotFound desc = could not find container \"a4754c31f4ee4b3378066adb6bb2ee88b912508f14405a39a46485225cd671a9\": container with ID starting with a4754c31f4ee4b3378066adb6bb2ee88b912508f14405a39a46485225cd671a9 not found: ID does not exist" May 21 11:34:57.669439 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.669360 2564 scope.go:117] "RemoveContainer" containerID="0f6ca201cb9fadc254b1b71366129284226ac9a24eab2669a170548ba6296773" May 21 11:34:57.669639 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.669618 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f6ca201cb9fadc254b1b71366129284226ac9a24eab2669a170548ba6296773"} err="failed to get container status \"0f6ca201cb9fadc254b1b71366129284226ac9a24eab2669a170548ba6296773\": rpc error: code = NotFound desc = could not find container \"0f6ca201cb9fadc254b1b71366129284226ac9a24eab2669a170548ba6296773\": container with ID starting with 0f6ca201cb9fadc254b1b71366129284226ac9a24eab2669a170548ba6296773 not found: ID does not exist" May 21 11:34:57.669718 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.669641 2564 scope.go:117] "RemoveContainer" containerID="64a000f48b85757d642db3a1bdd29ffee6fbae528cbcade8d4c4487579183ca9" May 21 11:34:57.669934 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.669913 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64a000f48b85757d642db3a1bdd29ffee6fbae528cbcade8d4c4487579183ca9"} err="failed to get container status \"64a000f48b85757d642db3a1bdd29ffee6fbae528cbcade8d4c4487579183ca9\": rpc error: code = NotFound desc = could not find container \"64a000f48b85757d642db3a1bdd29ffee6fbae528cbcade8d4c4487579183ca9\": container with ID starting with 64a000f48b85757d642db3a1bdd29ffee6fbae528cbcade8d4c4487579183ca9 not found: ID does not exist" May 21 11:34:57.670000 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.669937 2564 scope.go:117] "RemoveContainer" containerID="d53fd38ac99a7365dd743fd4ab2c5e797a2569a3efdbdbf2e4473aae0f42cc37" May 21 11:34:57.670235 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.670196 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d53fd38ac99a7365dd743fd4ab2c5e797a2569a3efdbdbf2e4473aae0f42cc37"} err="failed to get container status \"d53fd38ac99a7365dd743fd4ab2c5e797a2569a3efdbdbf2e4473aae0f42cc37\": rpc error: code = NotFound desc = could not find container \"d53fd38ac99a7365dd743fd4ab2c5e797a2569a3efdbdbf2e4473aae0f42cc37\": container with ID starting with d53fd38ac99a7365dd743fd4ab2c5e797a2569a3efdbdbf2e4473aae0f42cc37 not found: ID does not exist" May 21 11:34:57.670365 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.670352 2564 scope.go:117] "RemoveContainer" containerID="39ad7f7fc43e2206d95a5f5a64dde7b712992b31b11b6f066d4368c28ab1830b" May 21 11:34:57.670822 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.670798 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39ad7f7fc43e2206d95a5f5a64dde7b712992b31b11b6f066d4368c28ab1830b"} err="failed to get container status \"39ad7f7fc43e2206d95a5f5a64dde7b712992b31b11b6f066d4368c28ab1830b\": rpc error: code = NotFound desc = could not find container \"39ad7f7fc43e2206d95a5f5a64dde7b712992b31b11b6f066d4368c28ab1830b\": container with ID starting with 39ad7f7fc43e2206d95a5f5a64dde7b712992b31b11b6f066d4368c28ab1830b not found: ID does not exist" May 21 11:34:57.670822 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.670823 2564 scope.go:117] "RemoveContainer" containerID="017288a0b2183b29a5f37805d020ce9e1837f78d53036a287bc90230386ef9c1" May 21 11:34:57.671125 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.671102 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"017288a0b2183b29a5f37805d020ce9e1837f78d53036a287bc90230386ef9c1"} err="failed to get container status \"017288a0b2183b29a5f37805d020ce9e1837f78d53036a287bc90230386ef9c1\": rpc error: code = NotFound desc = could not find container \"017288a0b2183b29a5f37805d020ce9e1837f78d53036a287bc90230386ef9c1\": container with ID starting with 017288a0b2183b29a5f37805d020ce9e1837f78d53036a287bc90230386ef9c1 not found: ID does not exist" May 21 11:34:57.671199 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.671128 2564 scope.go:117] "RemoveContainer" containerID="a7855dd2768f1c61da22a69cf17c8ed0149658247a862f542a1dab54a8888a96" May 21 11:34:57.671344 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.671323 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7855dd2768f1c61da22a69cf17c8ed0149658247a862f542a1dab54a8888a96"} err="failed to get container status \"a7855dd2768f1c61da22a69cf17c8ed0149658247a862f542a1dab54a8888a96\": rpc error: code = NotFound desc = could not find container \"a7855dd2768f1c61da22a69cf17c8ed0149658247a862f542a1dab54a8888a96\": container with ID starting with a7855dd2768f1c61da22a69cf17c8ed0149658247a862f542a1dab54a8888a96 not found: ID does not exist" May 21 11:34:57.671423 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.671344 2564 scope.go:117] "RemoveContainer" containerID="a4754c31f4ee4b3378066adb6bb2ee88b912508f14405a39a46485225cd671a9" May 21 11:34:57.671610 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.671588 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4754c31f4ee4b3378066adb6bb2ee88b912508f14405a39a46485225cd671a9"} err="failed to get container status \"a4754c31f4ee4b3378066adb6bb2ee88b912508f14405a39a46485225cd671a9\": rpc error: code = NotFound desc = could not find container \"a4754c31f4ee4b3378066adb6bb2ee88b912508f14405a39a46485225cd671a9\": container with ID starting with a4754c31f4ee4b3378066adb6bb2ee88b912508f14405a39a46485225cd671a9 not found: ID does not exist" May 21 11:34:57.671662 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.671611 2564 scope.go:117] "RemoveContainer" containerID="0f6ca201cb9fadc254b1b71366129284226ac9a24eab2669a170548ba6296773" May 21 11:34:57.671830 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.671811 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f6ca201cb9fadc254b1b71366129284226ac9a24eab2669a170548ba6296773"} err="failed to get container status \"0f6ca201cb9fadc254b1b71366129284226ac9a24eab2669a170548ba6296773\": rpc error: code = NotFound desc = could not find container \"0f6ca201cb9fadc254b1b71366129284226ac9a24eab2669a170548ba6296773\": container with ID starting with 0f6ca201cb9fadc254b1b71366129284226ac9a24eab2669a170548ba6296773 not found: ID does not exist" May 21 11:34:57.671907 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.671833 2564 scope.go:117] "RemoveContainer" containerID="64a000f48b85757d642db3a1bdd29ffee6fbae528cbcade8d4c4487579183ca9" May 21 11:34:57.672395 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.672368 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64a000f48b85757d642db3a1bdd29ffee6fbae528cbcade8d4c4487579183ca9"} err="failed to get container status \"64a000f48b85757d642db3a1bdd29ffee6fbae528cbcade8d4c4487579183ca9\": rpc error: code = NotFound desc = could not find container \"64a000f48b85757d642db3a1bdd29ffee6fbae528cbcade8d4c4487579183ca9\": container with ID starting with 64a000f48b85757d642db3a1bdd29ffee6fbae528cbcade8d4c4487579183ca9 not found: ID does not exist" May 21 11:34:57.672498 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.672397 2564 scope.go:117] "RemoveContainer" containerID="d53fd38ac99a7365dd743fd4ab2c5e797a2569a3efdbdbf2e4473aae0f42cc37" May 21 11:34:57.672693 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.672644 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d53fd38ac99a7365dd743fd4ab2c5e797a2569a3efdbdbf2e4473aae0f42cc37"} err="failed to get container status \"d53fd38ac99a7365dd743fd4ab2c5e797a2569a3efdbdbf2e4473aae0f42cc37\": rpc error: code = NotFound desc = could not find container \"d53fd38ac99a7365dd743fd4ab2c5e797a2569a3efdbdbf2e4473aae0f42cc37\": container with ID starting with d53fd38ac99a7365dd743fd4ab2c5e797a2569a3efdbdbf2e4473aae0f42cc37 not found: ID does not exist" May 21 11:34:57.672693 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.672677 2564 scope.go:117] "RemoveContainer" containerID="39ad7f7fc43e2206d95a5f5a64dde7b712992b31b11b6f066d4368c28ab1830b" May 21 11:34:57.673000 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.672963 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39ad7f7fc43e2206d95a5f5a64dde7b712992b31b11b6f066d4368c28ab1830b"} err="failed to get container status \"39ad7f7fc43e2206d95a5f5a64dde7b712992b31b11b6f066d4368c28ab1830b\": rpc error: code = NotFound desc = could not find container \"39ad7f7fc43e2206d95a5f5a64dde7b712992b31b11b6f066d4368c28ab1830b\": container with ID starting with 39ad7f7fc43e2206d95a5f5a64dde7b712992b31b11b6f066d4368c28ab1830b not found: ID does not exist" May 21 11:34:57.673000 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.672994 2564 scope.go:117] "RemoveContainer" containerID="017288a0b2183b29a5f37805d020ce9e1837f78d53036a287bc90230386ef9c1" May 21 11:34:57.673257 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.673236 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"017288a0b2183b29a5f37805d020ce9e1837f78d53036a287bc90230386ef9c1"} err="failed to get container status \"017288a0b2183b29a5f37805d020ce9e1837f78d53036a287bc90230386ef9c1\": rpc error: code = NotFound desc = could not find container \"017288a0b2183b29a5f37805d020ce9e1837f78d53036a287bc90230386ef9c1\": container with ID starting with 017288a0b2183b29a5f37805d020ce9e1837f78d53036a287bc90230386ef9c1 not found: ID does not exist" May 21 11:34:57.673344 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.673259 2564 scope.go:117] "RemoveContainer" containerID="a7855dd2768f1c61da22a69cf17c8ed0149658247a862f542a1dab54a8888a96" May 21 11:34:57.673499 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.673480 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7855dd2768f1c61da22a69cf17c8ed0149658247a862f542a1dab54a8888a96"} err="failed to get container status \"a7855dd2768f1c61da22a69cf17c8ed0149658247a862f542a1dab54a8888a96\": rpc error: code = NotFound desc = could not find container \"a7855dd2768f1c61da22a69cf17c8ed0149658247a862f542a1dab54a8888a96\": container with ID starting with a7855dd2768f1c61da22a69cf17c8ed0149658247a862f542a1dab54a8888a96 not found: ID does not exist" May 21 11:34:57.673564 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.673501 2564 scope.go:117] "RemoveContainer" containerID="a4754c31f4ee4b3378066adb6bb2ee88b912508f14405a39a46485225cd671a9" May 21 11:34:57.673803 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.673776 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4754c31f4ee4b3378066adb6bb2ee88b912508f14405a39a46485225cd671a9"} err="failed to get container status \"a4754c31f4ee4b3378066adb6bb2ee88b912508f14405a39a46485225cd671a9\": rpc error: code = NotFound desc = could not find container \"a4754c31f4ee4b3378066adb6bb2ee88b912508f14405a39a46485225cd671a9\": container with ID starting with a4754c31f4ee4b3378066adb6bb2ee88b912508f14405a39a46485225cd671a9 not found: ID does not exist" May 21 11:34:57.673803 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.673800 2564 scope.go:117] "RemoveContainer" containerID="0f6ca201cb9fadc254b1b71366129284226ac9a24eab2669a170548ba6296773" May 21 11:34:57.674011 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.673994 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f6ca201cb9fadc254b1b71366129284226ac9a24eab2669a170548ba6296773"} err="failed to get container status \"0f6ca201cb9fadc254b1b71366129284226ac9a24eab2669a170548ba6296773\": rpc error: code = NotFound desc = could not find container \"0f6ca201cb9fadc254b1b71366129284226ac9a24eab2669a170548ba6296773\": container with ID starting with 0f6ca201cb9fadc254b1b71366129284226ac9a24eab2669a170548ba6296773 not found: ID does not exist" May 21 11:34:57.674011 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.674010 2564 scope.go:117] "RemoveContainer" containerID="64a000f48b85757d642db3a1bdd29ffee6fbae528cbcade8d4c4487579183ca9" May 21 11:34:57.674225 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.674202 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64a000f48b85757d642db3a1bdd29ffee6fbae528cbcade8d4c4487579183ca9"} err="failed to get container status \"64a000f48b85757d642db3a1bdd29ffee6fbae528cbcade8d4c4487579183ca9\": rpc error: code = NotFound desc = could not find container \"64a000f48b85757d642db3a1bdd29ffee6fbae528cbcade8d4c4487579183ca9\": container with ID starting with 64a000f48b85757d642db3a1bdd29ffee6fbae528cbcade8d4c4487579183ca9 not found: ID does not exist" May 21 11:34:57.674225 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.674224 2564 scope.go:117] "RemoveContainer" containerID="d53fd38ac99a7365dd743fd4ab2c5e797a2569a3efdbdbf2e4473aae0f42cc37" May 21 11:34:57.674465 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.674444 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d53fd38ac99a7365dd743fd4ab2c5e797a2569a3efdbdbf2e4473aae0f42cc37"} err="failed to get container status \"d53fd38ac99a7365dd743fd4ab2c5e797a2569a3efdbdbf2e4473aae0f42cc37\": rpc error: code = NotFound desc = could not find container \"d53fd38ac99a7365dd743fd4ab2c5e797a2569a3efdbdbf2e4473aae0f42cc37\": container with ID starting with d53fd38ac99a7365dd743fd4ab2c5e797a2569a3efdbdbf2e4473aae0f42cc37 not found: ID does not exist" May 21 11:34:57.674465 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.674465 2564 scope.go:117] "RemoveContainer" containerID="39ad7f7fc43e2206d95a5f5a64dde7b712992b31b11b6f066d4368c28ab1830b" May 21 11:34:57.674758 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.674732 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39ad7f7fc43e2206d95a5f5a64dde7b712992b31b11b6f066d4368c28ab1830b"} err="failed to get container status \"39ad7f7fc43e2206d95a5f5a64dde7b712992b31b11b6f066d4368c28ab1830b\": rpc error: code = NotFound desc = could not find container \"39ad7f7fc43e2206d95a5f5a64dde7b712992b31b11b6f066d4368c28ab1830b\": container with ID starting with 39ad7f7fc43e2206d95a5f5a64dde7b712992b31b11b6f066d4368c28ab1830b not found: ID does not exist" May 21 11:34:57.674827 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.674761 2564 scope.go:117] "RemoveContainer" containerID="017288a0b2183b29a5f37805d020ce9e1837f78d53036a287bc90230386ef9c1" May 21 11:34:57.675011 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.674989 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"017288a0b2183b29a5f37805d020ce9e1837f78d53036a287bc90230386ef9c1"} err="failed to get container status \"017288a0b2183b29a5f37805d020ce9e1837f78d53036a287bc90230386ef9c1\": rpc error: code = NotFound desc = could not find container \"017288a0b2183b29a5f37805d020ce9e1837f78d53036a287bc90230386ef9c1\": container with ID starting with 017288a0b2183b29a5f37805d020ce9e1837f78d53036a287bc90230386ef9c1 not found: ID does not exist" May 21 11:34:57.675065 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.675014 2564 scope.go:117] "RemoveContainer" containerID="a7855dd2768f1c61da22a69cf17c8ed0149658247a862f542a1dab54a8888a96" May 21 11:34:57.675275 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.675250 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7855dd2768f1c61da22a69cf17c8ed0149658247a862f542a1dab54a8888a96"} err="failed to get container status \"a7855dd2768f1c61da22a69cf17c8ed0149658247a862f542a1dab54a8888a96\": rpc error: code = NotFound desc = could not find container \"a7855dd2768f1c61da22a69cf17c8ed0149658247a862f542a1dab54a8888a96\": container with ID starting with a7855dd2768f1c61da22a69cf17c8ed0149658247a862f542a1dab54a8888a96 not found: ID does not exist" May 21 11:34:57.675343 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.675278 2564 scope.go:117] "RemoveContainer" containerID="a4754c31f4ee4b3378066adb6bb2ee88b912508f14405a39a46485225cd671a9" May 21 11:34:57.675495 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.675478 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4754c31f4ee4b3378066adb6bb2ee88b912508f14405a39a46485225cd671a9"} err="failed to get container status \"a4754c31f4ee4b3378066adb6bb2ee88b912508f14405a39a46485225cd671a9\": rpc error: code = NotFound desc = could not find container \"a4754c31f4ee4b3378066adb6bb2ee88b912508f14405a39a46485225cd671a9\": container with ID starting with a4754c31f4ee4b3378066adb6bb2ee88b912508f14405a39a46485225cd671a9 not found: ID does not exist" May 21 11:34:57.675550 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.675495 2564 scope.go:117] "RemoveContainer" containerID="0f6ca201cb9fadc254b1b71366129284226ac9a24eab2669a170548ba6296773" May 21 11:34:57.675716 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.675696 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f6ca201cb9fadc254b1b71366129284226ac9a24eab2669a170548ba6296773"} err="failed to get container status \"0f6ca201cb9fadc254b1b71366129284226ac9a24eab2669a170548ba6296773\": rpc error: code = NotFound desc = could not find container \"0f6ca201cb9fadc254b1b71366129284226ac9a24eab2669a170548ba6296773\": container with ID starting with 0f6ca201cb9fadc254b1b71366129284226ac9a24eab2669a170548ba6296773 not found: ID does not exist" May 21 11:34:57.675797 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.675718 2564 scope.go:117] "RemoveContainer" containerID="64a000f48b85757d642db3a1bdd29ffee6fbae528cbcade8d4c4487579183ca9" May 21 11:34:57.675962 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.675940 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64a000f48b85757d642db3a1bdd29ffee6fbae528cbcade8d4c4487579183ca9"} err="failed to get container status \"64a000f48b85757d642db3a1bdd29ffee6fbae528cbcade8d4c4487579183ca9\": rpc error: code = NotFound desc = could not find container \"64a000f48b85757d642db3a1bdd29ffee6fbae528cbcade8d4c4487579183ca9\": container with ID starting with 64a000f48b85757d642db3a1bdd29ffee6fbae528cbcade8d4c4487579183ca9 not found: ID does not exist" May 21 11:34:57.676028 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.675963 2564 scope.go:117] "RemoveContainer" containerID="d53fd38ac99a7365dd743fd4ab2c5e797a2569a3efdbdbf2e4473aae0f42cc37" May 21 11:34:57.676240 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.676216 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d53fd38ac99a7365dd743fd4ab2c5e797a2569a3efdbdbf2e4473aae0f42cc37"} err="failed to get container status \"d53fd38ac99a7365dd743fd4ab2c5e797a2569a3efdbdbf2e4473aae0f42cc37\": rpc error: code = NotFound desc = could not find container \"d53fd38ac99a7365dd743fd4ab2c5e797a2569a3efdbdbf2e4473aae0f42cc37\": container with ID starting with d53fd38ac99a7365dd743fd4ab2c5e797a2569a3efdbdbf2e4473aae0f42cc37 not found: ID does not exist" May 21 11:34:57.676307 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.676242 2564 scope.go:117] "RemoveContainer" containerID="39ad7f7fc43e2206d95a5f5a64dde7b712992b31b11b6f066d4368c28ab1830b" May 21 11:34:57.676810 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.676790 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39ad7f7fc43e2206d95a5f5a64dde7b712992b31b11b6f066d4368c28ab1830b"} err="failed to get container status \"39ad7f7fc43e2206d95a5f5a64dde7b712992b31b11b6f066d4368c28ab1830b\": rpc error: code = NotFound desc = could not find container \"39ad7f7fc43e2206d95a5f5a64dde7b712992b31b11b6f066d4368c28ab1830b\": container with ID starting with 39ad7f7fc43e2206d95a5f5a64dde7b712992b31b11b6f066d4368c28ab1830b not found: ID does not exist" May 21 11:34:57.676934 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.676812 2564 scope.go:117] "RemoveContainer" containerID="017288a0b2183b29a5f37805d020ce9e1837f78d53036a287bc90230386ef9c1" May 21 11:34:57.677119 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.677096 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"017288a0b2183b29a5f37805d020ce9e1837f78d53036a287bc90230386ef9c1"} err="failed to get container status \"017288a0b2183b29a5f37805d020ce9e1837f78d53036a287bc90230386ef9c1\": rpc error: code = NotFound desc = could not find container \"017288a0b2183b29a5f37805d020ce9e1837f78d53036a287bc90230386ef9c1\": container with ID starting with 017288a0b2183b29a5f37805d020ce9e1837f78d53036a287bc90230386ef9c1 not found: ID does not exist" May 21 11:34:57.677180 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.677121 2564 scope.go:117] "RemoveContainer" containerID="a7855dd2768f1c61da22a69cf17c8ed0149658247a862f542a1dab54a8888a96" May 21 11:34:57.677361 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.677338 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7855dd2768f1c61da22a69cf17c8ed0149658247a862f542a1dab54a8888a96"} err="failed to get container status \"a7855dd2768f1c61da22a69cf17c8ed0149658247a862f542a1dab54a8888a96\": rpc error: code = NotFound desc = could not find container \"a7855dd2768f1c61da22a69cf17c8ed0149658247a862f542a1dab54a8888a96\": container with ID starting with a7855dd2768f1c61da22a69cf17c8ed0149658247a862f542a1dab54a8888a96 not found: ID does not exist" May 21 11:34:57.677361 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.677361 2564 scope.go:117] "RemoveContainer" containerID="a4754c31f4ee4b3378066adb6bb2ee88b912508f14405a39a46485225cd671a9" May 21 11:34:57.677644 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.677618 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4754c31f4ee4b3378066adb6bb2ee88b912508f14405a39a46485225cd671a9"} err="failed to get container status \"a4754c31f4ee4b3378066adb6bb2ee88b912508f14405a39a46485225cd671a9\": rpc error: code = NotFound desc = could not find container \"a4754c31f4ee4b3378066adb6bb2ee88b912508f14405a39a46485225cd671a9\": container with ID starting with a4754c31f4ee4b3378066adb6bb2ee88b912508f14405a39a46485225cd671a9 not found: ID does not exist" May 21 11:34:57.684953 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.684925 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ebbf5734-045d-4cea-bf46-f6ac80a11d46" path="/var/lib/kubelet/pods/ebbf5734-045d-4cea-bf46-f6ac80a11d46/volumes" May 21 11:34:57.694353 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.694326 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/951519f5-fcca-4472-8f27-d025f50e6156-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"951519f5-fcca-4472-8f27-d025f50e6156\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:57.694478 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.694360 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/951519f5-fcca-4472-8f27-d025f50e6156-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"951519f5-fcca-4472-8f27-d025f50e6156\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:57.694697 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.694659 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/951519f5-fcca-4472-8f27-d025f50e6156-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"951519f5-fcca-4472-8f27-d025f50e6156\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:57.694821 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.694725 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/951519f5-fcca-4472-8f27-d025f50e6156-web-config\") pod \"prometheus-k8s-0\" (UID: \"951519f5-fcca-4472-8f27-d025f50e6156\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:57.694821 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.694758 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/951519f5-fcca-4472-8f27-d025f50e6156-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"951519f5-fcca-4472-8f27-d025f50e6156\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:57.694821 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.694788 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/951519f5-fcca-4472-8f27-d025f50e6156-config-out\") pod \"prometheus-k8s-0\" (UID: \"951519f5-fcca-4472-8f27-d025f50e6156\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:57.694981 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.694827 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/951519f5-fcca-4472-8f27-d025f50e6156-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"951519f5-fcca-4472-8f27-d025f50e6156\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:57.694981 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.694857 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/951519f5-fcca-4472-8f27-d025f50e6156-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"951519f5-fcca-4472-8f27-d025f50e6156\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:57.694981 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.694885 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/951519f5-fcca-4472-8f27-d025f50e6156-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"951519f5-fcca-4472-8f27-d025f50e6156\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:57.694981 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.694942 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/951519f5-fcca-4472-8f27-d025f50e6156-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"951519f5-fcca-4472-8f27-d025f50e6156\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:57.695192 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.695003 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/951519f5-fcca-4472-8f27-d025f50e6156-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"951519f5-fcca-4472-8f27-d025f50e6156\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:57.695192 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.695029 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/951519f5-fcca-4472-8f27-d025f50e6156-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"951519f5-fcca-4472-8f27-d025f50e6156\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:57.695299 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.695191 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/951519f5-fcca-4472-8f27-d025f50e6156-config\") pod \"prometheus-k8s-0\" (UID: \"951519f5-fcca-4472-8f27-d025f50e6156\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:57.695299 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.695227 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/951519f5-fcca-4472-8f27-d025f50e6156-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"951519f5-fcca-4472-8f27-d025f50e6156\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:57.695379 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.695319 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/951519f5-fcca-4472-8f27-d025f50e6156-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"951519f5-fcca-4472-8f27-d025f50e6156\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:57.695379 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.695360 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/951519f5-fcca-4472-8f27-d025f50e6156-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"951519f5-fcca-4472-8f27-d025f50e6156\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:57.695440 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.695417 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/951519f5-fcca-4472-8f27-d025f50e6156-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"951519f5-fcca-4472-8f27-d025f50e6156\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:57.695489 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.695435 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbjd7\" (UniqueName: \"kubernetes.io/projected/951519f5-fcca-4472-8f27-d025f50e6156-kube-api-access-dbjd7\") pod \"prometheus-k8s-0\" (UID: \"951519f5-fcca-4472-8f27-d025f50e6156\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:57.796202 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.796168 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/951519f5-fcca-4472-8f27-d025f50e6156-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"951519f5-fcca-4472-8f27-d025f50e6156\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:57.796202 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.796203 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/951519f5-fcca-4472-8f27-d025f50e6156-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"951519f5-fcca-4472-8f27-d025f50e6156\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:57.796443 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.796236 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/951519f5-fcca-4472-8f27-d025f50e6156-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"951519f5-fcca-4472-8f27-d025f50e6156\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:57.796443 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.796261 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dbjd7\" (UniqueName: \"kubernetes.io/projected/951519f5-fcca-4472-8f27-d025f50e6156-kube-api-access-dbjd7\") pod \"prometheus-k8s-0\" (UID: \"951519f5-fcca-4472-8f27-d025f50e6156\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:57.796443 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.796302 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/951519f5-fcca-4472-8f27-d025f50e6156-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"951519f5-fcca-4472-8f27-d025f50e6156\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:57.796443 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.796333 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/951519f5-fcca-4472-8f27-d025f50e6156-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"951519f5-fcca-4472-8f27-d025f50e6156\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:57.796443 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.796360 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/951519f5-fcca-4472-8f27-d025f50e6156-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"951519f5-fcca-4472-8f27-d025f50e6156\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:57.796443 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.796396 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/951519f5-fcca-4472-8f27-d025f50e6156-web-config\") pod \"prometheus-k8s-0\" (UID: \"951519f5-fcca-4472-8f27-d025f50e6156\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:57.796443 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.796428 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/951519f5-fcca-4472-8f27-d025f50e6156-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"951519f5-fcca-4472-8f27-d025f50e6156\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:57.796817 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.796470 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/951519f5-fcca-4472-8f27-d025f50e6156-config-out\") pod \"prometheus-k8s-0\" (UID: \"951519f5-fcca-4472-8f27-d025f50e6156\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:57.796817 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.796504 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/951519f5-fcca-4472-8f27-d025f50e6156-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"951519f5-fcca-4472-8f27-d025f50e6156\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:57.796817 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.796540 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/951519f5-fcca-4472-8f27-d025f50e6156-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"951519f5-fcca-4472-8f27-d025f50e6156\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:57.796817 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.796567 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/951519f5-fcca-4472-8f27-d025f50e6156-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"951519f5-fcca-4472-8f27-d025f50e6156\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:57.796817 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.796607 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/951519f5-fcca-4472-8f27-d025f50e6156-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"951519f5-fcca-4472-8f27-d025f50e6156\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:57.796817 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.796644 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/951519f5-fcca-4472-8f27-d025f50e6156-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"951519f5-fcca-4472-8f27-d025f50e6156\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:57.796817 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.796688 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/951519f5-fcca-4472-8f27-d025f50e6156-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"951519f5-fcca-4472-8f27-d025f50e6156\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:57.796817 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.796759 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/951519f5-fcca-4472-8f27-d025f50e6156-config\") pod \"prometheus-k8s-0\" (UID: \"951519f5-fcca-4472-8f27-d025f50e6156\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:57.796817 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.796796 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/951519f5-fcca-4472-8f27-d025f50e6156-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"951519f5-fcca-4472-8f27-d025f50e6156\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:57.797350 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.797134 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/951519f5-fcca-4472-8f27-d025f50e6156-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"951519f5-fcca-4472-8f27-d025f50e6156\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:57.797350 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.797148 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/951519f5-fcca-4472-8f27-d025f50e6156-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"951519f5-fcca-4472-8f27-d025f50e6156\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:57.800952 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.800652 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/951519f5-fcca-4472-8f27-d025f50e6156-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"951519f5-fcca-4472-8f27-d025f50e6156\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:57.800952 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.800800 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/951519f5-fcca-4472-8f27-d025f50e6156-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"951519f5-fcca-4472-8f27-d025f50e6156\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:57.800952 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.800814 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/951519f5-fcca-4472-8f27-d025f50e6156-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"951519f5-fcca-4472-8f27-d025f50e6156\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:57.800952 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.800942 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/951519f5-fcca-4472-8f27-d025f50e6156-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"951519f5-fcca-4472-8f27-d025f50e6156\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:57.801255 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.801103 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/951519f5-fcca-4472-8f27-d025f50e6156-web-config\") pod \"prometheus-k8s-0\" (UID: \"951519f5-fcca-4472-8f27-d025f50e6156\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:57.801378 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.801353 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/951519f5-fcca-4472-8f27-d025f50e6156-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"951519f5-fcca-4472-8f27-d025f50e6156\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:57.801484 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.801452 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/951519f5-fcca-4472-8f27-d025f50e6156-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"951519f5-fcca-4472-8f27-d025f50e6156\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:57.801885 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.801643 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/951519f5-fcca-4472-8f27-d025f50e6156-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"951519f5-fcca-4472-8f27-d025f50e6156\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:57.802904 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.802881 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/951519f5-fcca-4472-8f27-d025f50e6156-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"951519f5-fcca-4472-8f27-d025f50e6156\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:57.803323 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.803253 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/951519f5-fcca-4472-8f27-d025f50e6156-config\") pod \"prometheus-k8s-0\" (UID: \"951519f5-fcca-4472-8f27-d025f50e6156\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:57.803883 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.803537 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/951519f5-fcca-4472-8f27-d025f50e6156-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"951519f5-fcca-4472-8f27-d025f50e6156\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:57.803883 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.803838 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/951519f5-fcca-4472-8f27-d025f50e6156-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"951519f5-fcca-4472-8f27-d025f50e6156\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:57.804023 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.803966 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/951519f5-fcca-4472-8f27-d025f50e6156-config-out\") pod \"prometheus-k8s-0\" (UID: \"951519f5-fcca-4472-8f27-d025f50e6156\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:57.804023 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.803985 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/951519f5-fcca-4472-8f27-d025f50e6156-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"951519f5-fcca-4472-8f27-d025f50e6156\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:57.804761 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.804740 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/951519f5-fcca-4472-8f27-d025f50e6156-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"951519f5-fcca-4472-8f27-d025f50e6156\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:57.806354 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.806336 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbjd7\" (UniqueName: \"kubernetes.io/projected/951519f5-fcca-4472-8f27-d025f50e6156-kube-api-access-dbjd7\") pod \"prometheus-k8s-0\" (UID: \"951519f5-fcca-4472-8f27-d025f50e6156\") " pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:57.973031 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:57.972943 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" May 21 11:34:58.194037 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:58.193541 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] May 21 11:34:58.197842 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:34:58.197803 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod951519f5_fcca_4472_8f27_d025f50e6156.slice/crio-ef1cfc91cdc67a904702508153c5960baf6bb619a4bee3e3a4b1eef6e4a517e3 WatchSource:0}: Error finding container ef1cfc91cdc67a904702508153c5960baf6bb619a4bee3e3a4b1eef6e4a517e3: Status 404 returned error can't find the container with id ef1cfc91cdc67a904702508153c5960baf6bb619a4bee3e3a4b1eef6e4a517e3 May 21 11:34:58.577697 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:58.577659 2564 generic.go:358] "Generic (PLEG): container finished" podID="951519f5-fcca-4472-8f27-d025f50e6156" containerID="a70b155df8b11b84e99c680f66d9eac218cfb0421ec06ee64af1c7a144bf1573" exitCode=0 May 21 11:34:58.578158 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:58.577745 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"951519f5-fcca-4472-8f27-d025f50e6156","Type":"ContainerDied","Data":"a70b155df8b11b84e99c680f66d9eac218cfb0421ec06ee64af1c7a144bf1573"} May 21 11:34:58.578158 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:58.577784 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"951519f5-fcca-4472-8f27-d025f50e6156","Type":"ContainerStarted","Data":"ef1cfc91cdc67a904702508153c5960baf6bb619a4bee3e3a4b1eef6e4a517e3"} May 21 11:34:58.580276 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:58.580249 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xf5gc" event={"ID":"414e12fe-b9ac-4bbf-8470-ad442215f5cb","Type":"ContainerStarted","Data":"8dcb72829d6d58d29c51f48922d79c2f86ff8fad91afeae96f2599313437a0b9"} May 21 11:34:58.580378 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:58.580282 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xf5gc" event={"ID":"414e12fe-b9ac-4bbf-8470-ad442215f5cb","Type":"ContainerStarted","Data":"379d23a9698491b2dd13b0103a1b9ce6ccd4f959020311124b078749cf982b51"} May 21 11:34:58.622334 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:58.622291 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-xf5gc" podStartSLOduration=252.599238162 podStartE2EDuration="4m13.622277035s" podCreationTimestamp="2026-05-21 11:30:45 +0000 UTC" firstStartedPulling="2026-05-21 11:34:57.021372816 +0000 UTC m=+251.916656353" lastFinishedPulling="2026-05-21 11:34:58.044411691 +0000 UTC m=+252.939695226" observedRunningTime="2026-05-21 11:34:58.62163826 +0000 UTC m=+253.516921809" watchObservedRunningTime="2026-05-21 11:34:58.622277035 +0000 UTC m=+253.517560580" May 21 11:34:59.592265 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:59.592226 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"951519f5-fcca-4472-8f27-d025f50e6156","Type":"ContainerStarted","Data":"a0d1b8c622b8e898d8e8905bdc3fd9e580cb55785a70d1fb29a454843a48aee5"} May 21 11:34:59.592652 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:59.592273 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"951519f5-fcca-4472-8f27-d025f50e6156","Type":"ContainerStarted","Data":"8aa15af60d0fe603bf32090ddb5f2eb796f8ed4ccccd8d3b955930a62cdf4ed7"} May 21 11:34:59.592652 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:59.592283 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"951519f5-fcca-4472-8f27-d025f50e6156","Type":"ContainerStarted","Data":"dab4b916c82d9501a58414601b7dbf849ca28eeb51ab8578e6520e9e0db95115"} May 21 11:34:59.592652 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:59.592292 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"951519f5-fcca-4472-8f27-d025f50e6156","Type":"ContainerStarted","Data":"e52949115969a95f91f81407a5fa8e05fbb1e2920d61f7b3251e4abf7144851f"} May 21 11:34:59.592652 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:59.592300 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"951519f5-fcca-4472-8f27-d025f50e6156","Type":"ContainerStarted","Data":"f52b81ba9c41a0b79394dc4d2afc6215841679b0dd51653d69bb0034dba5d33e"} May 21 11:34:59.592652 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:59.592307 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"951519f5-fcca-4472-8f27-d025f50e6156","Type":"ContainerStarted","Data":"696e6c0502cfa14ad05e5d2eba79cd3d02090e1352c768f88fc8280338d1c134"} May 21 11:34:59.625936 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:34:59.625860 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.625841218 podStartE2EDuration="2.625841218s" podCreationTimestamp="2026-05-21 11:34:57 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-21 11:34:59.622181068 +0000 UTC m=+254.517464632" watchObservedRunningTime="2026-05-21 11:34:59.625841218 +0000 UTC m=+254.521124764" May 21 11:35:02.973879 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:35:02.973826 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" May 21 11:35:45.563352 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:35:45.563322 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-6g9xk_be463222-9e99-47cb-bf6b-525913ca9e29/console-operator/2.log" May 21 11:35:45.564461 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:35:45.563361 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-6g9xk_be463222-9e99-47cb-bf6b-525913ca9e29/console-operator/2.log" May 21 11:35:45.566830 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:35:45.566806 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sr2wt_45027dd2-2a4d-4ff5-b33f-48e3ee052186/ovn-acl-logging/0.log" May 21 11:35:45.566999 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:35:45.566980 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sr2wt_45027dd2-2a4d-4ff5-b33f-48e3ee052186/ovn-acl-logging/0.log" May 21 11:35:45.573721 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:35:45.573702 2564 kubelet.go:1628] "Image garbage collection succeeded" May 21 11:35:57.974036 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:35:57.973995 2564 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" May 21 11:35:57.990649 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:35:57.990626 2564 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" May 21 11:35:58.783259 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:35:58.783230 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" May 21 11:36:32.885497 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:36:32.885418 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-nlrhz"] May 21 11:36:32.889458 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:36:32.889430 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nlrhz" May 21 11:36:32.892441 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:36:32.892419 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" May 21 11:36:32.895851 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:36:32.895825 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-nlrhz"] May 21 11:36:32.913405 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:36:32.913373 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6b372f05-30c2-4e48-b411-9e1941308c2a-original-pull-secret\") pod \"global-pull-secret-syncer-nlrhz\" (UID: \"6b372f05-30c2-4e48-b411-9e1941308c2a\") " pod="kube-system/global-pull-secret-syncer-nlrhz" May 21 11:36:32.913559 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:36:32.913417 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6b372f05-30c2-4e48-b411-9e1941308c2a-dbus\") pod \"global-pull-secret-syncer-nlrhz\" (UID: \"6b372f05-30c2-4e48-b411-9e1941308c2a\") " pod="kube-system/global-pull-secret-syncer-nlrhz" May 21 11:36:32.913559 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:36:32.913451 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6b372f05-30c2-4e48-b411-9e1941308c2a-kubelet-config\") pod \"global-pull-secret-syncer-nlrhz\" (UID: \"6b372f05-30c2-4e48-b411-9e1941308c2a\") " pod="kube-system/global-pull-secret-syncer-nlrhz" May 21 11:36:33.014095 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:36:33.014056 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6b372f05-30c2-4e48-b411-9e1941308c2a-original-pull-secret\") pod \"global-pull-secret-syncer-nlrhz\" (UID: \"6b372f05-30c2-4e48-b411-9e1941308c2a\") " pod="kube-system/global-pull-secret-syncer-nlrhz" May 21 11:36:33.014270 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:36:33.014122 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6b372f05-30c2-4e48-b411-9e1941308c2a-dbus\") pod \"global-pull-secret-syncer-nlrhz\" (UID: \"6b372f05-30c2-4e48-b411-9e1941308c2a\") " pod="kube-system/global-pull-secret-syncer-nlrhz" May 21 11:36:33.014270 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:36:33.014151 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6b372f05-30c2-4e48-b411-9e1941308c2a-kubelet-config\") pod \"global-pull-secret-syncer-nlrhz\" (UID: \"6b372f05-30c2-4e48-b411-9e1941308c2a\") " pod="kube-system/global-pull-secret-syncer-nlrhz" May 21 11:36:33.014270 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:36:33.014260 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6b372f05-30c2-4e48-b411-9e1941308c2a-kubelet-config\") pod \"global-pull-secret-syncer-nlrhz\" (UID: \"6b372f05-30c2-4e48-b411-9e1941308c2a\") " pod="kube-system/global-pull-secret-syncer-nlrhz" May 21 11:36:33.014438 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:36:33.014315 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6b372f05-30c2-4e48-b411-9e1941308c2a-dbus\") pod \"global-pull-secret-syncer-nlrhz\" (UID: \"6b372f05-30c2-4e48-b411-9e1941308c2a\") " pod="kube-system/global-pull-secret-syncer-nlrhz" May 21 11:36:33.016418 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:36:33.016398 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6b372f05-30c2-4e48-b411-9e1941308c2a-original-pull-secret\") pod \"global-pull-secret-syncer-nlrhz\" (UID: \"6b372f05-30c2-4e48-b411-9e1941308c2a\") " pod="kube-system/global-pull-secret-syncer-nlrhz" May 21 11:36:33.200172 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:36:33.200053 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nlrhz" May 21 11:36:33.331744 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:36:33.331718 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-nlrhz"] May 21 11:36:33.334233 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:36:33.334202 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b372f05_30c2_4e48_b411_9e1941308c2a.slice/crio-167f747f4708abaff438ca9ee082b5e603fff9c29dba4e9dd9b9bf413a9c97d1 WatchSource:0}: Error finding container 167f747f4708abaff438ca9ee082b5e603fff9c29dba4e9dd9b9bf413a9c97d1: Status 404 returned error can't find the container with id 167f747f4708abaff438ca9ee082b5e603fff9c29dba4e9dd9b9bf413a9c97d1 May 21 11:36:33.336180 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:36:33.336163 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 21 11:36:33.879983 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:36:33.879944 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-nlrhz" event={"ID":"6b372f05-30c2-4e48-b411-9e1941308c2a","Type":"ContainerStarted","Data":"167f747f4708abaff438ca9ee082b5e603fff9c29dba4e9dd9b9bf413a9c97d1"} May 21 11:36:37.897043 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:36:37.897003 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-nlrhz" event={"ID":"6b372f05-30c2-4e48-b411-9e1941308c2a","Type":"ContainerStarted","Data":"76eedd55f70c9d9a7cc8243cbb7a4f21327b3ab1da7f37ff7562651d28da4c93"} May 21 11:36:37.919282 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:36:37.919235 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-nlrhz" podStartSLOduration=2.018038583 podStartE2EDuration="5.91922064s" podCreationTimestamp="2026-05-21 11:36:32 +0000 UTC" firstStartedPulling="2026-05-21 11:36:33.336293329 +0000 UTC m=+348.231576854" lastFinishedPulling="2026-05-21 11:36:37.237475386 +0000 UTC m=+352.132758911" observedRunningTime="2026-05-21 11:36:37.917458707 +0000 UTC m=+352.812742254" watchObservedRunningTime="2026-05-21 11:36:37.91922064 +0000 UTC m=+352.814504184" May 21 11:37:52.396036 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:37:52.396000 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-79c8d999ff-p2wm2"] May 21 11:37:52.398252 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:37:52.398235 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-79c8d999ff-p2wm2" May 21 11:37:52.401563 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:37:52.401537 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" May 21 11:37:52.401685 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:37:52.401546 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" May 21 11:37:52.401685 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:37:52.401677 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-9tppg\"" May 21 11:37:52.406361 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:37:52.406336 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-79c8d999ff-p2wm2"] May 21 11:37:52.483985 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:37:52.483947 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3002ba24-6fc7-4375-8909-47775178c10d-bound-sa-token\") pod \"cert-manager-79c8d999ff-p2wm2\" (UID: \"3002ba24-6fc7-4375-8909-47775178c10d\") " pod="cert-manager/cert-manager-79c8d999ff-p2wm2" May 21 11:37:52.484195 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:37:52.484009 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbd45\" (UniqueName: \"kubernetes.io/projected/3002ba24-6fc7-4375-8909-47775178c10d-kube-api-access-wbd45\") pod \"cert-manager-79c8d999ff-p2wm2\" (UID: \"3002ba24-6fc7-4375-8909-47775178c10d\") " pod="cert-manager/cert-manager-79c8d999ff-p2wm2" May 21 11:37:52.584590 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:37:52.584555 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3002ba24-6fc7-4375-8909-47775178c10d-bound-sa-token\") pod \"cert-manager-79c8d999ff-p2wm2\" (UID: \"3002ba24-6fc7-4375-8909-47775178c10d\") " pod="cert-manager/cert-manager-79c8d999ff-p2wm2" May 21 11:37:52.584756 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:37:52.584612 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wbd45\" (UniqueName: \"kubernetes.io/projected/3002ba24-6fc7-4375-8909-47775178c10d-kube-api-access-wbd45\") pod \"cert-manager-79c8d999ff-p2wm2\" (UID: \"3002ba24-6fc7-4375-8909-47775178c10d\") " pod="cert-manager/cert-manager-79c8d999ff-p2wm2" May 21 11:37:52.593069 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:37:52.593037 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3002ba24-6fc7-4375-8909-47775178c10d-bound-sa-token\") pod \"cert-manager-79c8d999ff-p2wm2\" (UID: \"3002ba24-6fc7-4375-8909-47775178c10d\") " pod="cert-manager/cert-manager-79c8d999ff-p2wm2" May 21 11:37:52.593228 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:37:52.593210 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbd45\" (UniqueName: \"kubernetes.io/projected/3002ba24-6fc7-4375-8909-47775178c10d-kube-api-access-wbd45\") pod \"cert-manager-79c8d999ff-p2wm2\" (UID: \"3002ba24-6fc7-4375-8909-47775178c10d\") " pod="cert-manager/cert-manager-79c8d999ff-p2wm2" May 21 11:37:52.721760 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:37:52.721661 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-79c8d999ff-p2wm2" May 21 11:37:52.852384 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:37:52.852360 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-79c8d999ff-p2wm2"] May 21 11:37:52.854826 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:37:52.854796 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3002ba24_6fc7_4375_8909_47775178c10d.slice/crio-810f9d3a2d1a99dadf6382abeafa12f1bf7396d84ce2e60ca30c54a515e531ac WatchSource:0}: Error finding container 810f9d3a2d1a99dadf6382abeafa12f1bf7396d84ce2e60ca30c54a515e531ac: Status 404 returned error can't find the container with id 810f9d3a2d1a99dadf6382abeafa12f1bf7396d84ce2e60ca30c54a515e531ac May 21 11:37:53.127606 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:37:53.127575 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-79c8d999ff-p2wm2" event={"ID":"3002ba24-6fc7-4375-8909-47775178c10d","Type":"ContainerStarted","Data":"810f9d3a2d1a99dadf6382abeafa12f1bf7396d84ce2e60ca30c54a515e531ac"} May 21 11:37:56.142149 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:37:56.142115 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-79c8d999ff-p2wm2" event={"ID":"3002ba24-6fc7-4375-8909-47775178c10d","Type":"ContainerStarted","Data":"62ffc9817bbcf37bd959a4eb96aec32aa408d9c2762015051ba5748f03e5fc54"} May 21 11:37:56.159662 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:37:56.159574 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager/cert-manager-79c8d999ff-p2wm2" podStartSLOduration=1.111222959 podStartE2EDuration="4.159558221s" podCreationTimestamp="2026-05-21 11:37:52 +0000 UTC" firstStartedPulling="2026-05-21 11:37:52.856667339 +0000 UTC m=+427.751950873" lastFinishedPulling="2026-05-21 11:37:55.905002613 +0000 UTC m=+430.800286135" observedRunningTime="2026-05-21 11:37:56.158539684 +0000 UTC m=+431.053823223" watchObservedRunningTime="2026-05-21 11:37:56.159558221 +0000 UTC m=+431.054841768" May 21 11:38:00.419558 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:38:00.419515 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5fd9cb47f5-pvxcn"] May 21 11:38:00.422930 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:38:00.422912 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5fd9cb47f5-pvxcn" May 21 11:38:00.426149 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:38:00.426126 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" May 21 11:38:00.427493 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:38:00.427474 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" May 21 11:38:00.427737 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:38:00.427708 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" May 21 11:38:00.427737 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:38:00.427711 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-7z46j\"" May 21 11:38:00.427906 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:38:00.427768 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" May 21 11:38:00.427906 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:38:00.427815 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" May 21 11:38:00.441472 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:38:00.441447 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5fd9cb47f5-pvxcn"] May 21 11:38:00.555026 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:38:00.554990 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbx6c\" (UniqueName: \"kubernetes.io/projected/1bfd0a23-d936-4c65-a519-77218cd5f6f3-kube-api-access-nbx6c\") pod \"lws-controller-manager-5fd9cb47f5-pvxcn\" (UID: \"1bfd0a23-d936-4c65-a519-77218cd5f6f3\") " pod="openshift-lws-operator/lws-controller-manager-5fd9cb47f5-pvxcn" May 21 11:38:00.555237 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:38:00.555040 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1bfd0a23-d936-4c65-a519-77218cd5f6f3-metrics-cert\") pod \"lws-controller-manager-5fd9cb47f5-pvxcn\" (UID: \"1bfd0a23-d936-4c65-a519-77218cd5f6f3\") " pod="openshift-lws-operator/lws-controller-manager-5fd9cb47f5-pvxcn" May 21 11:38:00.555237 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:38:00.555061 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1bfd0a23-d936-4c65-a519-77218cd5f6f3-cert\") pod \"lws-controller-manager-5fd9cb47f5-pvxcn\" (UID: \"1bfd0a23-d936-4c65-a519-77218cd5f6f3\") " pod="openshift-lws-operator/lws-controller-manager-5fd9cb47f5-pvxcn" May 21 11:38:00.555237 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:38:00.555145 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1bfd0a23-d936-4c65-a519-77218cd5f6f3-manager-config\") pod \"lws-controller-manager-5fd9cb47f5-pvxcn\" (UID: \"1bfd0a23-d936-4c65-a519-77218cd5f6f3\") " pod="openshift-lws-operator/lws-controller-manager-5fd9cb47f5-pvxcn" May 21 11:38:00.656135 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:38:00.656067 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nbx6c\" (UniqueName: \"kubernetes.io/projected/1bfd0a23-d936-4c65-a519-77218cd5f6f3-kube-api-access-nbx6c\") pod \"lws-controller-manager-5fd9cb47f5-pvxcn\" (UID: \"1bfd0a23-d936-4c65-a519-77218cd5f6f3\") " pod="openshift-lws-operator/lws-controller-manager-5fd9cb47f5-pvxcn" May 21 11:38:00.656335 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:38:00.656157 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1bfd0a23-d936-4c65-a519-77218cd5f6f3-metrics-cert\") pod \"lws-controller-manager-5fd9cb47f5-pvxcn\" (UID: \"1bfd0a23-d936-4c65-a519-77218cd5f6f3\") " pod="openshift-lws-operator/lws-controller-manager-5fd9cb47f5-pvxcn" May 21 11:38:00.656335 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:38:00.656179 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1bfd0a23-d936-4c65-a519-77218cd5f6f3-cert\") pod \"lws-controller-manager-5fd9cb47f5-pvxcn\" (UID: \"1bfd0a23-d936-4c65-a519-77218cd5f6f3\") " pod="openshift-lws-operator/lws-controller-manager-5fd9cb47f5-pvxcn" May 21 11:38:00.656335 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:38:00.656225 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1bfd0a23-d936-4c65-a519-77218cd5f6f3-manager-config\") pod \"lws-controller-manager-5fd9cb47f5-pvxcn\" (UID: \"1bfd0a23-d936-4c65-a519-77218cd5f6f3\") " pod="openshift-lws-operator/lws-controller-manager-5fd9cb47f5-pvxcn" May 21 11:38:00.656849 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:38:00.656827 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1bfd0a23-d936-4c65-a519-77218cd5f6f3-manager-config\") pod \"lws-controller-manager-5fd9cb47f5-pvxcn\" (UID: \"1bfd0a23-d936-4c65-a519-77218cd5f6f3\") " pod="openshift-lws-operator/lws-controller-manager-5fd9cb47f5-pvxcn" May 21 11:38:00.658878 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:38:00.658858 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1bfd0a23-d936-4c65-a519-77218cd5f6f3-cert\") pod \"lws-controller-manager-5fd9cb47f5-pvxcn\" (UID: \"1bfd0a23-d936-4c65-a519-77218cd5f6f3\") " pod="openshift-lws-operator/lws-controller-manager-5fd9cb47f5-pvxcn" May 21 11:38:00.658964 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:38:00.658891 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1bfd0a23-d936-4c65-a519-77218cd5f6f3-metrics-cert\") pod \"lws-controller-manager-5fd9cb47f5-pvxcn\" (UID: \"1bfd0a23-d936-4c65-a519-77218cd5f6f3\") " pod="openshift-lws-operator/lws-controller-manager-5fd9cb47f5-pvxcn" May 21 11:38:00.668643 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:38:00.668623 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbx6c\" (UniqueName: \"kubernetes.io/projected/1bfd0a23-d936-4c65-a519-77218cd5f6f3-kube-api-access-nbx6c\") pod \"lws-controller-manager-5fd9cb47f5-pvxcn\" (UID: \"1bfd0a23-d936-4c65-a519-77218cd5f6f3\") " pod="openshift-lws-operator/lws-controller-manager-5fd9cb47f5-pvxcn" May 21 11:38:00.732070 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:38:00.732000 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5fd9cb47f5-pvxcn" May 21 11:38:00.861223 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:38:00.861197 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5fd9cb47f5-pvxcn"] May 21 11:38:00.863218 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:38:00.863193 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bfd0a23_d936_4c65_a519_77218cd5f6f3.slice/crio-60f91a50e764d738b10456b4e425683e57b110af7c9e1bc188279c959081699d WatchSource:0}: Error finding container 60f91a50e764d738b10456b4e425683e57b110af7c9e1bc188279c959081699d: Status 404 returned error can't find the container with id 60f91a50e764d738b10456b4e425683e57b110af7c9e1bc188279c959081699d May 21 11:38:01.161190 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:38:01.161160 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5fd9cb47f5-pvxcn" event={"ID":"1bfd0a23-d936-4c65-a519-77218cd5f6f3","Type":"ContainerStarted","Data":"60f91a50e764d738b10456b4e425683e57b110af7c9e1bc188279c959081699d"} May 21 11:38:04.171372 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:38:04.171287 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5fd9cb47f5-pvxcn" event={"ID":"1bfd0a23-d936-4c65-a519-77218cd5f6f3","Type":"ContainerStarted","Data":"f0aa54c0c0cfd03f2946de12e1f77d8a9f6b4c8cf59969f64a5f6edd230dae9f"} May 21 11:38:04.171813 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:38:04.171423 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5fd9cb47f5-pvxcn" May 21 11:38:04.191029 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:38:04.190977 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5fd9cb47f5-pvxcn" podStartSLOduration=1.244081623 podStartE2EDuration="4.19096196s" podCreationTimestamp="2026-05-21 11:38:00 +0000 UTC" firstStartedPulling="2026-05-21 11:38:00.865156352 +0000 UTC m=+435.760439878" lastFinishedPulling="2026-05-21 11:38:03.81203669 +0000 UTC m=+438.707320215" observedRunningTime="2026-05-21 11:38:04.188229474 +0000 UTC m=+439.083513017" watchObservedRunningTime="2026-05-21 11:38:04.19096196 +0000 UTC m=+439.086245575" May 21 11:38:15.176594 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:38:15.176561 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5fd9cb47f5-pvxcn" May 21 11:38:22.184003 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:38:22.183964 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7c4fb87789-ppglg"] May 21 11:38:22.187503 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:38:22.187487 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7c4fb87789-ppglg" May 21 11:38:22.192640 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:38:22.192616 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" May 21 11:38:22.192640 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:38:22.192632 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" May 21 11:38:22.192839 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:38:22.192728 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" May 21 11:38:22.192981 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:38:22.192963 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-5xvcl\"" May 21 11:38:22.193229 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:38:22.193213 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" May 21 11:38:22.207446 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:38:22.207422 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7c4fb87789-ppglg"] May 21 11:38:22.246138 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:38:22.246105 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8v6jf\" (UniqueName: \"kubernetes.io/projected/52ff6c3d-2aae-4a8f-ba40-78b0afe91aa8-kube-api-access-8v6jf\") pod \"opendatahub-operator-controller-manager-7c4fb87789-ppglg\" (UID: \"52ff6c3d-2aae-4a8f-ba40-78b0afe91aa8\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4fb87789-ppglg" May 21 11:38:22.246275 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:38:22.246161 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/52ff6c3d-2aae-4a8f-ba40-78b0afe91aa8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c4fb87789-ppglg\" (UID: \"52ff6c3d-2aae-4a8f-ba40-78b0afe91aa8\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4fb87789-ppglg" May 21 11:38:22.246275 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:38:22.246254 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/52ff6c3d-2aae-4a8f-ba40-78b0afe91aa8-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c4fb87789-ppglg\" (UID: \"52ff6c3d-2aae-4a8f-ba40-78b0afe91aa8\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4fb87789-ppglg" May 21 11:38:22.347461 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:38:22.347425 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/52ff6c3d-2aae-4a8f-ba40-78b0afe91aa8-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c4fb87789-ppglg\" (UID: \"52ff6c3d-2aae-4a8f-ba40-78b0afe91aa8\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4fb87789-ppglg" May 21 11:38:22.347461 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:38:22.347463 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8v6jf\" (UniqueName: \"kubernetes.io/projected/52ff6c3d-2aae-4a8f-ba40-78b0afe91aa8-kube-api-access-8v6jf\") pod \"opendatahub-operator-controller-manager-7c4fb87789-ppglg\" (UID: \"52ff6c3d-2aae-4a8f-ba40-78b0afe91aa8\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4fb87789-ppglg" May 21 11:38:22.347660 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:38:22.347503 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/52ff6c3d-2aae-4a8f-ba40-78b0afe91aa8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c4fb87789-ppglg\" (UID: \"52ff6c3d-2aae-4a8f-ba40-78b0afe91aa8\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4fb87789-ppglg" May 21 11:38:22.350010 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:38:22.349981 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/52ff6c3d-2aae-4a8f-ba40-78b0afe91aa8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c4fb87789-ppglg\" (UID: \"52ff6c3d-2aae-4a8f-ba40-78b0afe91aa8\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4fb87789-ppglg" May 21 11:38:22.350143 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:38:22.350033 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/52ff6c3d-2aae-4a8f-ba40-78b0afe91aa8-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c4fb87789-ppglg\" (UID: \"52ff6c3d-2aae-4a8f-ba40-78b0afe91aa8\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4fb87789-ppglg" May 21 11:38:22.365601 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:38:22.365575 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8v6jf\" (UniqueName: \"kubernetes.io/projected/52ff6c3d-2aae-4a8f-ba40-78b0afe91aa8-kube-api-access-8v6jf\") pod \"opendatahub-operator-controller-manager-7c4fb87789-ppglg\" (UID: \"52ff6c3d-2aae-4a8f-ba40-78b0afe91aa8\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4fb87789-ppglg" May 21 11:38:22.497542 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:38:22.497449 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7c4fb87789-ppglg" May 21 11:38:22.633158 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:38:22.633038 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7c4fb87789-ppglg"] May 21 11:38:22.635820 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:38:22.635789 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52ff6c3d_2aae_4a8f_ba40_78b0afe91aa8.slice/crio-75a59a8394ed12460c29a85df5fc997f18011bd51d70966067c54a8fa377ebf1 WatchSource:0}: Error finding container 75a59a8394ed12460c29a85df5fc997f18011bd51d70966067c54a8fa377ebf1: Status 404 returned error can't find the container with id 75a59a8394ed12460c29a85df5fc997f18011bd51d70966067c54a8fa377ebf1 May 21 11:38:23.229943 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:38:23.229905 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c4fb87789-ppglg" event={"ID":"52ff6c3d-2aae-4a8f-ba40-78b0afe91aa8","Type":"ContainerStarted","Data":"75a59a8394ed12460c29a85df5fc997f18011bd51d70966067c54a8fa377ebf1"} May 21 11:38:25.239426 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:38:25.239385 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c4fb87789-ppglg" event={"ID":"52ff6c3d-2aae-4a8f-ba40-78b0afe91aa8","Type":"ContainerStarted","Data":"a4ad092eb51277ec8df9ae8298706e4e7f1cbdac77f56fdeacb7f73541854b02"} May 21 11:38:25.239925 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:38:25.239508 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7c4fb87789-ppglg" May 21 11:38:25.271262 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:38:25.271215 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7c4fb87789-ppglg" podStartSLOduration=0.780976881 podStartE2EDuration="3.271199666s" podCreationTimestamp="2026-05-21 11:38:22 +0000 UTC" firstStartedPulling="2026-05-21 11:38:22.637628534 +0000 UTC m=+457.532912057" lastFinishedPulling="2026-05-21 11:38:25.127851316 +0000 UTC m=+460.023134842" observedRunningTime="2026-05-21 11:38:25.268907867 +0000 UTC m=+460.164191413" watchObservedRunningTime="2026-05-21 11:38:25.271199666 +0000 UTC m=+460.166483211" May 21 11:38:36.245990 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:38:36.245957 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7c4fb87789-ppglg" May 21 11:39:08.739588 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:39:08.739556 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-df5b64c7d-w6jp2"] May 21 11:39:08.746437 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:39:08.746416 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-df5b64c7d-w6jp2" May 21 11:39:08.751256 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:39:08.751229 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" May 21 11:39:08.751256 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:39:08.751247 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-97tnl\"" May 21 11:39:08.753265 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:39:08.753242 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" May 21 11:39:08.765125 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:39:08.765099 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-df5b64c7d-w6jp2"] May 21 11:39:08.855230 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:39:08.855193 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5a2c2383-4064-47e5-9c12-32c47b6ef502-tmp\") pod \"kube-auth-proxy-df5b64c7d-w6jp2\" (UID: \"5a2c2383-4064-47e5-9c12-32c47b6ef502\") " pod="openshift-ingress/kube-auth-proxy-df5b64c7d-w6jp2" May 21 11:39:08.855425 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:39:08.855255 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a2c2383-4064-47e5-9c12-32c47b6ef502-tls-certs\") pod \"kube-auth-proxy-df5b64c7d-w6jp2\" (UID: \"5a2c2383-4064-47e5-9c12-32c47b6ef502\") " pod="openshift-ingress/kube-auth-proxy-df5b64c7d-w6jp2" May 21 11:39:08.855425 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:39:08.855347 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cjw9\" (UniqueName: \"kubernetes.io/projected/5a2c2383-4064-47e5-9c12-32c47b6ef502-kube-api-access-8cjw9\") pod \"kube-auth-proxy-df5b64c7d-w6jp2\" (UID: \"5a2c2383-4064-47e5-9c12-32c47b6ef502\") " pod="openshift-ingress/kube-auth-proxy-df5b64c7d-w6jp2" May 21 11:39:08.956467 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:39:08.956430 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5a2c2383-4064-47e5-9c12-32c47b6ef502-tmp\") pod \"kube-auth-proxy-df5b64c7d-w6jp2\" (UID: \"5a2c2383-4064-47e5-9c12-32c47b6ef502\") " pod="openshift-ingress/kube-auth-proxy-df5b64c7d-w6jp2" May 21 11:39:08.956634 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:39:08.956478 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a2c2383-4064-47e5-9c12-32c47b6ef502-tls-certs\") pod \"kube-auth-proxy-df5b64c7d-w6jp2\" (UID: \"5a2c2383-4064-47e5-9c12-32c47b6ef502\") " pod="openshift-ingress/kube-auth-proxy-df5b64c7d-w6jp2" May 21 11:39:08.956634 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:39:08.956538 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8cjw9\" (UniqueName: \"kubernetes.io/projected/5a2c2383-4064-47e5-9c12-32c47b6ef502-kube-api-access-8cjw9\") pod \"kube-auth-proxy-df5b64c7d-w6jp2\" (UID: \"5a2c2383-4064-47e5-9c12-32c47b6ef502\") " pod="openshift-ingress/kube-auth-proxy-df5b64c7d-w6jp2" May 21 11:39:08.958790 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:39:08.958765 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5a2c2383-4064-47e5-9c12-32c47b6ef502-tmp\") pod \"kube-auth-proxy-df5b64c7d-w6jp2\" (UID: \"5a2c2383-4064-47e5-9c12-32c47b6ef502\") " pod="openshift-ingress/kube-auth-proxy-df5b64c7d-w6jp2" May 21 11:39:08.958980 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:39:08.958962 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a2c2383-4064-47e5-9c12-32c47b6ef502-tls-certs\") pod \"kube-auth-proxy-df5b64c7d-w6jp2\" (UID: \"5a2c2383-4064-47e5-9c12-32c47b6ef502\") " pod="openshift-ingress/kube-auth-proxy-df5b64c7d-w6jp2" May 21 11:39:08.967638 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:39:08.967618 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cjw9\" (UniqueName: \"kubernetes.io/projected/5a2c2383-4064-47e5-9c12-32c47b6ef502-kube-api-access-8cjw9\") pod \"kube-auth-proxy-df5b64c7d-w6jp2\" (UID: \"5a2c2383-4064-47e5-9c12-32c47b6ef502\") " pod="openshift-ingress/kube-auth-proxy-df5b64c7d-w6jp2" May 21 11:39:09.056313 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:39:09.056289 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-df5b64c7d-w6jp2" May 21 11:39:09.188154 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:39:09.188125 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-df5b64c7d-w6jp2"] May 21 11:39:09.190353 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:39:09.190326 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a2c2383_4064_47e5_9c12_32c47b6ef502.slice/crio-2d86fdf40cad3e56adb2c13b89ce80a14ee2dff51f2170d054364c498d3fb7f3 WatchSource:0}: Error finding container 2d86fdf40cad3e56adb2c13b89ce80a14ee2dff51f2170d054364c498d3fb7f3: Status 404 returned error can't find the container with id 2d86fdf40cad3e56adb2c13b89ce80a14ee2dff51f2170d054364c498d3fb7f3 May 21 11:39:09.383881 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:39:09.383792 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-df5b64c7d-w6jp2" event={"ID":"5a2c2383-4064-47e5-9c12-32c47b6ef502","Type":"ContainerStarted","Data":"2d86fdf40cad3e56adb2c13b89ce80a14ee2dff51f2170d054364c498d3fb7f3"} May 21 11:39:13.400196 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:39:13.400158 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-df5b64c7d-w6jp2" event={"ID":"5a2c2383-4064-47e5-9c12-32c47b6ef502","Type":"ContainerStarted","Data":"d453b26107f80ac6c47e0e8bf65a3cdfee5231e23494093bdecb5d00da7a737f"} May 21 11:39:13.420226 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:39:13.420174 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-df5b64c7d-w6jp2" podStartSLOduration=1.9216570050000001 podStartE2EDuration="5.420159178s" podCreationTimestamp="2026-05-21 11:39:08 +0000 UTC" firstStartedPulling="2026-05-21 11:39:09.192108313 +0000 UTC m=+504.087391840" lastFinishedPulling="2026-05-21 11:39:12.690610491 +0000 UTC m=+507.585894013" observedRunningTime="2026-05-21 11:39:13.417649625 +0000 UTC m=+508.312933173" watchObservedRunningTime="2026-05-21 11:39:13.420159178 +0000 UTC m=+508.315442744" May 21 11:40:45.590459 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:40:45.590428 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-6g9xk_be463222-9e99-47cb-bf6b-525913ca9e29/console-operator/2.log" May 21 11:40:45.591594 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:40:45.591570 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-6g9xk_be463222-9e99-47cb-bf6b-525913ca9e29/console-operator/2.log" May 21 11:40:45.593913 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:40:45.593891 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sr2wt_45027dd2-2a4d-4ff5-b33f-48e3ee052186/ovn-acl-logging/0.log" May 21 11:40:45.595122 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:40:45.595097 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sr2wt_45027dd2-2a4d-4ff5-b33f-48e3ee052186/ovn-acl-logging/0.log" May 21 11:43:27.400758 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:43:27.400721 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-q48pq"] May 21 11:43:27.404136 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:43:27.404117 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-q48pq" May 21 11:43:27.407597 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:43:27.407576 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" May 21 11:43:27.407684 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:43:27.407586 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" May 21 11:43:27.408932 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:43:27.408913 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" May 21 11:43:27.409021 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:43:27.408913 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-626b4\"" May 21 11:43:27.413498 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:43:27.413475 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-q48pq"] May 21 11:43:27.495461 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:43:27.495431 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-q48pq"] May 21 11:43:27.541673 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:43:27.541634 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/246978ea-469a-4ae1-8020-05d8f894ef90-config-file\") pod \"limitador-limitador-7d549b5b-q48pq\" (UID: \"246978ea-469a-4ae1-8020-05d8f894ef90\") " pod="kuadrant-system/limitador-limitador-7d549b5b-q48pq" May 21 11:43:27.541840 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:43:27.541738 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjj8w\" (UniqueName: \"kubernetes.io/projected/246978ea-469a-4ae1-8020-05d8f894ef90-kube-api-access-pjj8w\") pod \"limitador-limitador-7d549b5b-q48pq\" (UID: \"246978ea-469a-4ae1-8020-05d8f894ef90\") " pod="kuadrant-system/limitador-limitador-7d549b5b-q48pq" May 21 11:43:27.642721 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:43:27.642681 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/246978ea-469a-4ae1-8020-05d8f894ef90-config-file\") pod \"limitador-limitador-7d549b5b-q48pq\" (UID: \"246978ea-469a-4ae1-8020-05d8f894ef90\") " pod="kuadrant-system/limitador-limitador-7d549b5b-q48pq" May 21 11:43:27.642867 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:43:27.642758 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pjj8w\" (UniqueName: \"kubernetes.io/projected/246978ea-469a-4ae1-8020-05d8f894ef90-kube-api-access-pjj8w\") pod \"limitador-limitador-7d549b5b-q48pq\" (UID: \"246978ea-469a-4ae1-8020-05d8f894ef90\") " pod="kuadrant-system/limitador-limitador-7d549b5b-q48pq" May 21 11:43:27.643306 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:43:27.643285 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/246978ea-469a-4ae1-8020-05d8f894ef90-config-file\") pod \"limitador-limitador-7d549b5b-q48pq\" (UID: \"246978ea-469a-4ae1-8020-05d8f894ef90\") " pod="kuadrant-system/limitador-limitador-7d549b5b-q48pq" May 21 11:43:27.652309 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:43:27.652249 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjj8w\" (UniqueName: \"kubernetes.io/projected/246978ea-469a-4ae1-8020-05d8f894ef90-kube-api-access-pjj8w\") pod \"limitador-limitador-7d549b5b-q48pq\" (UID: \"246978ea-469a-4ae1-8020-05d8f894ef90\") " pod="kuadrant-system/limitador-limitador-7d549b5b-q48pq" May 21 11:43:27.715408 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:43:27.715375 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-q48pq" May 21 11:43:27.858808 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:43:27.858774 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-q48pq"] May 21 11:43:27.862006 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:43:27.861969 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod246978ea_469a_4ae1_8020_05d8f894ef90.slice/crio-503f293ad03dbfeec0d83737773537d2d1969d7ca6b15e0a79c018034d339e7e WatchSource:0}: Error finding container 503f293ad03dbfeec0d83737773537d2d1969d7ca6b15e0a79c018034d339e7e: Status 404 returned error can't find the container with id 503f293ad03dbfeec0d83737773537d2d1969d7ca6b15e0a79c018034d339e7e May 21 11:43:27.863804 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:43:27.863785 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 21 11:43:28.277706 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:43:28.277672 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-q48pq" event={"ID":"246978ea-469a-4ae1-8020-05d8f894ef90","Type":"ContainerStarted","Data":"503f293ad03dbfeec0d83737773537d2d1969d7ca6b15e0a79c018034d339e7e"} May 21 11:43:31.292481 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:43:31.292448 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-q48pq" event={"ID":"246978ea-469a-4ae1-8020-05d8f894ef90","Type":"ContainerStarted","Data":"6899d8a99a2e6255b6ed7942298050bf5a7686b0c61f2ed62d37b56c30852ae8"} May 21 11:43:31.292872 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:43:31.292505 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-7d549b5b-q48pq" May 21 11:43:31.312196 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:43:31.312150 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-7d549b5b-q48pq" podStartSLOduration=1.633722524 podStartE2EDuration="4.312136086s" podCreationTimestamp="2026-05-21 11:43:27 +0000 UTC" firstStartedPulling="2026-05-21 11:43:27.863947694 +0000 UTC m=+762.759231221" lastFinishedPulling="2026-05-21 11:43:30.542361251 +0000 UTC m=+765.437644783" observedRunningTime="2026-05-21 11:43:31.310257081 +0000 UTC m=+766.205540626" watchObservedRunningTime="2026-05-21 11:43:31.312136086 +0000 UTC m=+766.207419659" May 21 11:43:42.297215 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:43:42.297182 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-7d549b5b-q48pq" May 21 11:43:45.495271 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:43:45.495236 2564 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-q48pq"] May 21 11:43:45.495743 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:43:45.495486 2564 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-7d549b5b-q48pq" podUID="246978ea-469a-4ae1-8020-05d8f894ef90" containerName="limitador" containerID="cri-o://6899d8a99a2e6255b6ed7942298050bf5a7686b0c61f2ed62d37b56c30852ae8" gracePeriod=30 May 21 11:43:46.053249 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:43:46.053223 2564 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-q48pq" May 21 11:43:46.101748 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:43:46.101669 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/246978ea-469a-4ae1-8020-05d8f894ef90-config-file\") pod \"246978ea-469a-4ae1-8020-05d8f894ef90\" (UID: \"246978ea-469a-4ae1-8020-05d8f894ef90\") " May 21 11:43:46.101906 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:43:46.101753 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pjj8w\" (UniqueName: \"kubernetes.io/projected/246978ea-469a-4ae1-8020-05d8f894ef90-kube-api-access-pjj8w\") pod \"246978ea-469a-4ae1-8020-05d8f894ef90\" (UID: \"246978ea-469a-4ae1-8020-05d8f894ef90\") " May 21 11:43:46.102026 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:43:46.102001 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/246978ea-469a-4ae1-8020-05d8f894ef90-config-file" (OuterVolumeSpecName: "config-file") pod "246978ea-469a-4ae1-8020-05d8f894ef90" (UID: "246978ea-469a-4ae1-8020-05d8f894ef90"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 21 11:43:46.103940 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:43:46.103912 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/246978ea-469a-4ae1-8020-05d8f894ef90-kube-api-access-pjj8w" (OuterVolumeSpecName: "kube-api-access-pjj8w") pod "246978ea-469a-4ae1-8020-05d8f894ef90" (UID: "246978ea-469a-4ae1-8020-05d8f894ef90"). InnerVolumeSpecName "kube-api-access-pjj8w". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 21 11:43:46.203327 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:43:46.203280 2564 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/246978ea-469a-4ae1-8020-05d8f894ef90-config-file\") on node \"ip-10-0-139-213.ec2.internal\" DevicePath \"\"" May 21 11:43:46.203327 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:43:46.203321 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pjj8w\" (UniqueName: \"kubernetes.io/projected/246978ea-469a-4ae1-8020-05d8f894ef90-kube-api-access-pjj8w\") on node \"ip-10-0-139-213.ec2.internal\" DevicePath \"\"" May 21 11:43:46.345007 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:43:46.344966 2564 generic.go:358] "Generic (PLEG): container finished" podID="246978ea-469a-4ae1-8020-05d8f894ef90" containerID="6899d8a99a2e6255b6ed7942298050bf5a7686b0c61f2ed62d37b56c30852ae8" exitCode=0 May 21 11:43:46.345184 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:43:46.345033 2564 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-q48pq" May 21 11:43:46.345184 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:43:46.345046 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-q48pq" event={"ID":"246978ea-469a-4ae1-8020-05d8f894ef90","Type":"ContainerDied","Data":"6899d8a99a2e6255b6ed7942298050bf5a7686b0c61f2ed62d37b56c30852ae8"} May 21 11:43:46.345184 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:43:46.345112 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-q48pq" event={"ID":"246978ea-469a-4ae1-8020-05d8f894ef90","Type":"ContainerDied","Data":"503f293ad03dbfeec0d83737773537d2d1969d7ca6b15e0a79c018034d339e7e"} May 21 11:43:46.345184 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:43:46.345128 2564 scope.go:117] "RemoveContainer" containerID="6899d8a99a2e6255b6ed7942298050bf5a7686b0c61f2ed62d37b56c30852ae8" May 21 11:43:46.353984 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:43:46.353963 2564 scope.go:117] "RemoveContainer" containerID="6899d8a99a2e6255b6ed7942298050bf5a7686b0c61f2ed62d37b56c30852ae8" May 21 11:43:46.354273 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:43:46.354253 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6899d8a99a2e6255b6ed7942298050bf5a7686b0c61f2ed62d37b56c30852ae8\": container with ID starting with 6899d8a99a2e6255b6ed7942298050bf5a7686b0c61f2ed62d37b56c30852ae8 not found: ID does not exist" containerID="6899d8a99a2e6255b6ed7942298050bf5a7686b0c61f2ed62d37b56c30852ae8" May 21 11:43:46.354342 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:43:46.354283 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6899d8a99a2e6255b6ed7942298050bf5a7686b0c61f2ed62d37b56c30852ae8"} err="failed to get container status \"6899d8a99a2e6255b6ed7942298050bf5a7686b0c61f2ed62d37b56c30852ae8\": rpc error: code = NotFound desc = could not find container \"6899d8a99a2e6255b6ed7942298050bf5a7686b0c61f2ed62d37b56c30852ae8\": container with ID starting with 6899d8a99a2e6255b6ed7942298050bf5a7686b0c61f2ed62d37b56c30852ae8 not found: ID does not exist" May 21 11:43:46.366480 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:43:46.366456 2564 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-q48pq"] May 21 11:43:46.370225 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:43:46.370204 2564 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-q48pq"] May 21 11:43:47.684023 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:43:47.683989 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="246978ea-469a-4ae1-8020-05d8f894ef90" path="/var/lib/kubelet/pods/246978ea-469a-4ae1-8020-05d8f894ef90/volumes" May 21 11:45:45.616377 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:45:45.616292 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-6g9xk_be463222-9e99-47cb-bf6b-525913ca9e29/console-operator/2.log" May 21 11:45:45.618900 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:45:45.618873 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-6g9xk_be463222-9e99-47cb-bf6b-525913ca9e29/console-operator/2.log" May 21 11:45:45.619845 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:45:45.619823 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sr2wt_45027dd2-2a4d-4ff5-b33f-48e3ee052186/ovn-acl-logging/0.log" May 21 11:45:45.621932 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:45:45.621914 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sr2wt_45027dd2-2a4d-4ff5-b33f-48e3ee052186/ovn-acl-logging/0.log" May 21 11:48:32.530288 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:32.530196 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-68fb5d86bc-mnw6m"] May 21 11:48:32.530719 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:32.530554 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="246978ea-469a-4ae1-8020-05d8f894ef90" containerName="limitador" May 21 11:48:32.530719 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:32.530564 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="246978ea-469a-4ae1-8020-05d8f894ef90" containerName="limitador" May 21 11:48:32.530719 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:32.530625 2564 memory_manager.go:356] "RemoveStaleState removing state" podUID="246978ea-469a-4ae1-8020-05d8f894ef90" containerName="limitador" May 21 11:48:32.533498 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:32.533478 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68fb5d86bc-mnw6m" May 21 11:48:32.536422 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:32.536400 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-2tc9w\"" May 21 11:48:32.536534 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:32.536399 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" May 21 11:48:32.544191 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:32.544171 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-68fb5d86bc-mnw6m"] May 21 11:48:32.691022 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:32.690984 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8k88\" (UniqueName: \"kubernetes.io/projected/2ad47427-3743-4ab5-8b53-5abec3da61b6-kube-api-access-l8k88\") pod \"maas-controller-68fb5d86bc-mnw6m\" (UID: \"2ad47427-3743-4ab5-8b53-5abec3da61b6\") " pod="opendatahub/maas-controller-68fb5d86bc-mnw6m" May 21 11:48:32.792242 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:32.792159 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-l8k88\" (UniqueName: \"kubernetes.io/projected/2ad47427-3743-4ab5-8b53-5abec3da61b6-kube-api-access-l8k88\") pod \"maas-controller-68fb5d86bc-mnw6m\" (UID: \"2ad47427-3743-4ab5-8b53-5abec3da61b6\") " pod="opendatahub/maas-controller-68fb5d86bc-mnw6m" May 21 11:48:32.801963 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:32.801938 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8k88\" (UniqueName: \"kubernetes.io/projected/2ad47427-3743-4ab5-8b53-5abec3da61b6-kube-api-access-l8k88\") pod \"maas-controller-68fb5d86bc-mnw6m\" (UID: \"2ad47427-3743-4ab5-8b53-5abec3da61b6\") " pod="opendatahub/maas-controller-68fb5d86bc-mnw6m" May 21 11:48:32.844923 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:32.844891 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68fb5d86bc-mnw6m" May 21 11:48:32.982984 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:32.982886 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-68fb5d86bc-mnw6m"] May 21 11:48:32.985914 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:48:32.985875 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ad47427_3743_4ab5_8b53_5abec3da61b6.slice/crio-ac5d33fe8d17a5c179d7e8afb197b95b7049931432cce3eeb427f825cee10067 WatchSource:0}: Error finding container ac5d33fe8d17a5c179d7e8afb197b95b7049931432cce3eeb427f825cee10067: Status 404 returned error can't find the container with id ac5d33fe8d17a5c179d7e8afb197b95b7049931432cce3eeb427f825cee10067 May 21 11:48:32.987528 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:32.987511 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 21 11:48:33.327187 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:33.327148 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68fb5d86bc-mnw6m" event={"ID":"2ad47427-3743-4ab5-8b53-5abec3da61b6","Type":"ContainerStarted","Data":"ac5d33fe8d17a5c179d7e8afb197b95b7049931432cce3eeb427f825cee10067"} May 21 11:48:34.019334 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:34.019293 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-d4c56f87b-fxl25"] May 21 11:48:34.023002 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:34.022984 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d4c56f87b-fxl25" May 21 11:48:34.032782 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:34.032755 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d4c56f87b-fxl25"] May 21 11:48:34.104959 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:34.104869 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mwzk\" (UniqueName: \"kubernetes.io/projected/416ae4bf-c10a-4ae7-b5a6-8fdeec864c29-kube-api-access-2mwzk\") pod \"maas-controller-d4c56f87b-fxl25\" (UID: \"416ae4bf-c10a-4ae7-b5a6-8fdeec864c29\") " pod="opendatahub/maas-controller-d4c56f87b-fxl25" May 21 11:48:34.206371 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:34.206334 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2mwzk\" (UniqueName: \"kubernetes.io/projected/416ae4bf-c10a-4ae7-b5a6-8fdeec864c29-kube-api-access-2mwzk\") pod \"maas-controller-d4c56f87b-fxl25\" (UID: \"416ae4bf-c10a-4ae7-b5a6-8fdeec864c29\") " pod="opendatahub/maas-controller-d4c56f87b-fxl25" May 21 11:48:34.216666 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:34.216632 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mwzk\" (UniqueName: \"kubernetes.io/projected/416ae4bf-c10a-4ae7-b5a6-8fdeec864c29-kube-api-access-2mwzk\") pod \"maas-controller-d4c56f87b-fxl25\" (UID: \"416ae4bf-c10a-4ae7-b5a6-8fdeec864c29\") " pod="opendatahub/maas-controller-d4c56f87b-fxl25" May 21 11:48:34.337660 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:34.337623 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d4c56f87b-fxl25" May 21 11:48:35.370204 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:35.370174 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d4c56f87b-fxl25"] May 21 11:48:35.372666 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:48:35.372638 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod416ae4bf_c10a_4ae7_b5a6_8fdeec864c29.slice/crio-142f3d6ca6b131bda4772164471a440c6ea80e7e3881dcc5ca7cec3475e1099a WatchSource:0}: Error finding container 142f3d6ca6b131bda4772164471a440c6ea80e7e3881dcc5ca7cec3475e1099a: Status 404 returned error can't find the container with id 142f3d6ca6b131bda4772164471a440c6ea80e7e3881dcc5ca7cec3475e1099a May 21 11:48:36.241471 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:36.241438 2564 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-68fb5d86bc-mnw6m"] May 21 11:48:36.270282 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:36.270208 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7f6fcccfc8-7bzvn"] May 21 11:48:36.274176 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:36.274152 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f6fcccfc8-7bzvn" May 21 11:48:36.287259 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:36.287231 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7f6fcccfc8-7bzvn"] May 21 11:48:36.343934 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:36.343896 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68fb5d86bc-mnw6m" event={"ID":"2ad47427-3743-4ab5-8b53-5abec3da61b6","Type":"ContainerStarted","Data":"bf507f5f24b612930444a5ea12a6eb47ee075d69d8173568e7b63d864cb5cab5"} May 21 11:48:36.344153 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:36.344026 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-68fb5d86bc-mnw6m" May 21 11:48:36.345044 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:36.345014 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d4c56f87b-fxl25" event={"ID":"416ae4bf-c10a-4ae7-b5a6-8fdeec864c29","Type":"ContainerStarted","Data":"142f3d6ca6b131bda4772164471a440c6ea80e7e3881dcc5ca7cec3475e1099a"} May 21 11:48:36.362686 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:36.362639 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-68fb5d86bc-mnw6m" podStartSLOduration=2.055702508 podStartE2EDuration="4.362622165s" podCreationTimestamp="2026-05-21 11:48:32 +0000 UTC" firstStartedPulling="2026-05-21 11:48:32.987636267 +0000 UTC m=+1067.882919791" lastFinishedPulling="2026-05-21 11:48:35.294555925 +0000 UTC m=+1070.189839448" observedRunningTime="2026-05-21 11:48:36.360035634 +0000 UTC m=+1071.255319180" watchObservedRunningTime="2026-05-21 11:48:36.362622165 +0000 UTC m=+1071.257905712" May 21 11:48:36.429924 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:36.429891 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4n4m\" (UniqueName: \"kubernetes.io/projected/55a65d66-020b-4e51-b189-f6dc806cc77c-kube-api-access-p4n4m\") pod \"maas-controller-7f6fcccfc8-7bzvn\" (UID: \"55a65d66-020b-4e51-b189-f6dc806cc77c\") " pod="opendatahub/maas-controller-7f6fcccfc8-7bzvn" May 21 11:48:36.531142 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:36.531036 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p4n4m\" (UniqueName: \"kubernetes.io/projected/55a65d66-020b-4e51-b189-f6dc806cc77c-kube-api-access-p4n4m\") pod \"maas-controller-7f6fcccfc8-7bzvn\" (UID: \"55a65d66-020b-4e51-b189-f6dc806cc77c\") " pod="opendatahub/maas-controller-7f6fcccfc8-7bzvn" May 21 11:48:36.540767 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:36.540735 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4n4m\" (UniqueName: \"kubernetes.io/projected/55a65d66-020b-4e51-b189-f6dc806cc77c-kube-api-access-p4n4m\") pod \"maas-controller-7f6fcccfc8-7bzvn\" (UID: \"55a65d66-020b-4e51-b189-f6dc806cc77c\") " pod="opendatahub/maas-controller-7f6fcccfc8-7bzvn" May 21 11:48:36.588541 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:36.588510 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f6fcccfc8-7bzvn" May 21 11:48:36.736017 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:36.735993 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7f6fcccfc8-7bzvn"] May 21 11:48:36.737687 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:48:36.737654 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55a65d66_020b_4e51_b189_f6dc806cc77c.slice/crio-c2fa53ed1fac18ba8ab5ba322a54186cb1862c98977add00ba29ba0c80a7867b WatchSource:0}: Error finding container c2fa53ed1fac18ba8ab5ba322a54186cb1862c98977add00ba29ba0c80a7867b: Status 404 returned error can't find the container with id c2fa53ed1fac18ba8ab5ba322a54186cb1862c98977add00ba29ba0c80a7867b May 21 11:48:37.350875 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:37.350840 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f6fcccfc8-7bzvn" event={"ID":"55a65d66-020b-4e51-b189-f6dc806cc77c","Type":"ContainerStarted","Data":"c2fa53ed1fac18ba8ab5ba322a54186cb1862c98977add00ba29ba0c80a7867b"} May 21 11:48:37.351058 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:37.351037 2564 kuberuntime_container.go:864] "Killing container with a grace period" pod="opendatahub/maas-controller-68fb5d86bc-mnw6m" podUID="2ad47427-3743-4ab5-8b53-5abec3da61b6" containerName="manager" containerID="cri-o://bf507f5f24b612930444a5ea12a6eb47ee075d69d8173568e7b63d864cb5cab5" gracePeriod=30 May 21 11:48:37.598445 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:37.598423 2564 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68fb5d86bc-mnw6m" May 21 11:48:37.742197 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:37.742154 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l8k88\" (UniqueName: \"kubernetes.io/projected/2ad47427-3743-4ab5-8b53-5abec3da61b6-kube-api-access-l8k88\") pod \"2ad47427-3743-4ab5-8b53-5abec3da61b6\" (UID: \"2ad47427-3743-4ab5-8b53-5abec3da61b6\") " May 21 11:48:37.744542 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:37.744514 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ad47427-3743-4ab5-8b53-5abec3da61b6-kube-api-access-l8k88" (OuterVolumeSpecName: "kube-api-access-l8k88") pod "2ad47427-3743-4ab5-8b53-5abec3da61b6" (UID: "2ad47427-3743-4ab5-8b53-5abec3da61b6"). InnerVolumeSpecName "kube-api-access-l8k88". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 21 11:48:37.843670 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:37.843594 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l8k88\" (UniqueName: \"kubernetes.io/projected/2ad47427-3743-4ab5-8b53-5abec3da61b6-kube-api-access-l8k88\") on node \"ip-10-0-139-213.ec2.internal\" DevicePath \"\"" May 21 11:48:38.355921 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:38.355882 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f6fcccfc8-7bzvn" event={"ID":"55a65d66-020b-4e51-b189-f6dc806cc77c","Type":"ContainerStarted","Data":"27ac2f932a99c4acf0b7036d9cf9f4459e63cbba4434322265666bfa3f1428da"} May 21 11:48:38.356106 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:38.355994 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7f6fcccfc8-7bzvn" May 21 11:48:38.357123 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:38.357101 2564 generic.go:358] "Generic (PLEG): container finished" podID="2ad47427-3743-4ab5-8b53-5abec3da61b6" containerID="bf507f5f24b612930444a5ea12a6eb47ee075d69d8173568e7b63d864cb5cab5" exitCode=0 May 21 11:48:38.357206 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:38.357165 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68fb5d86bc-mnw6m" event={"ID":"2ad47427-3743-4ab5-8b53-5abec3da61b6","Type":"ContainerDied","Data":"bf507f5f24b612930444a5ea12a6eb47ee075d69d8173568e7b63d864cb5cab5"} May 21 11:48:38.357206 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:38.357194 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68fb5d86bc-mnw6m" event={"ID":"2ad47427-3743-4ab5-8b53-5abec3da61b6","Type":"ContainerDied","Data":"ac5d33fe8d17a5c179d7e8afb197b95b7049931432cce3eeb427f825cee10067"} May 21 11:48:38.357302 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:38.357245 2564 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68fb5d86bc-mnw6m" May 21 11:48:38.357344 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:38.357218 2564 scope.go:117] "RemoveContainer" containerID="bf507f5f24b612930444a5ea12a6eb47ee075d69d8173568e7b63d864cb5cab5" May 21 11:48:38.358730 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:38.358708 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d4c56f87b-fxl25" event={"ID":"416ae4bf-c10a-4ae7-b5a6-8fdeec864c29","Type":"ContainerStarted","Data":"86a481e9f533480ae36459cf471380a4a5eab8b72a0672bde96d4f7a53460dff"} May 21 11:48:38.369995 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:38.369977 2564 scope.go:117] "RemoveContainer" containerID="bf507f5f24b612930444a5ea12a6eb47ee075d69d8173568e7b63d864cb5cab5" May 21 11:48:38.370286 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:48:38.370264 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf507f5f24b612930444a5ea12a6eb47ee075d69d8173568e7b63d864cb5cab5\": container with ID starting with bf507f5f24b612930444a5ea12a6eb47ee075d69d8173568e7b63d864cb5cab5 not found: ID does not exist" containerID="bf507f5f24b612930444a5ea12a6eb47ee075d69d8173568e7b63d864cb5cab5" May 21 11:48:38.370342 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:38.370298 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf507f5f24b612930444a5ea12a6eb47ee075d69d8173568e7b63d864cb5cab5"} err="failed to get container status \"bf507f5f24b612930444a5ea12a6eb47ee075d69d8173568e7b63d864cb5cab5\": rpc error: code = NotFound desc = could not find container \"bf507f5f24b612930444a5ea12a6eb47ee075d69d8173568e7b63d864cb5cab5\": container with ID starting with bf507f5f24b612930444a5ea12a6eb47ee075d69d8173568e7b63d864cb5cab5 not found: ID does not exist" May 21 11:48:38.371745 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:38.371637 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-7f6fcccfc8-7bzvn" podStartSLOduration=1.813572823 podStartE2EDuration="2.371622514s" podCreationTimestamp="2026-05-21 11:48:36 +0000 UTC" firstStartedPulling="2026-05-21 11:48:36.739256877 +0000 UTC m=+1071.634540403" lastFinishedPulling="2026-05-21 11:48:37.297306567 +0000 UTC m=+1072.192590094" observedRunningTime="2026-05-21 11:48:38.371124015 +0000 UTC m=+1073.266407561" watchObservedRunningTime="2026-05-21 11:48:38.371622514 +0000 UTC m=+1073.266906061" May 21 11:48:38.383863 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:38.383842 2564 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-68fb5d86bc-mnw6m"] May 21 11:48:38.387624 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:38.387602 2564 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-68fb5d86bc-mnw6m"] May 21 11:48:38.403304 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:38.403260 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-d4c56f87b-fxl25" podStartSLOduration=2.480520593 podStartE2EDuration="4.403250269s" podCreationTimestamp="2026-05-21 11:48:34 +0000 UTC" firstStartedPulling="2026-05-21 11:48:35.374088061 +0000 UTC m=+1070.269371598" lastFinishedPulling="2026-05-21 11:48:37.296817751 +0000 UTC m=+1072.192101274" observedRunningTime="2026-05-21 11:48:38.401816734 +0000 UTC m=+1073.297100279" watchObservedRunningTime="2026-05-21 11:48:38.403250269 +0000 UTC m=+1073.298533813" May 21 11:48:39.364209 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:39.364176 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-d4c56f87b-fxl25" May 21 11:48:39.684220 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:39.684142 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ad47427-3743-4ab5-8b53-5abec3da61b6" path="/var/lib/kubelet/pods/2ad47427-3743-4ab5-8b53-5abec3da61b6/volumes" May 21 11:48:49.369699 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:49.369661 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7f6fcccfc8-7bzvn" May 21 11:48:49.415277 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:49.415244 2564 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-d4c56f87b-fxl25"] May 21 11:48:49.415509 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:49.415470 2564 kuberuntime_container.go:864] "Killing container with a grace period" pod="opendatahub/maas-controller-d4c56f87b-fxl25" podUID="416ae4bf-c10a-4ae7-b5a6-8fdeec864c29" containerName="manager" containerID="cri-o://86a481e9f533480ae36459cf471380a4a5eab8b72a0672bde96d4f7a53460dff" gracePeriod=30 May 21 11:48:49.424420 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:49.424394 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-d4c56f87b-fxl25" May 21 11:48:49.667834 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:49.667811 2564 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d4c56f87b-fxl25" May 21 11:48:49.753490 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:49.748353 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2mwzk\" (UniqueName: \"kubernetes.io/projected/416ae4bf-c10a-4ae7-b5a6-8fdeec864c29-kube-api-access-2mwzk\") pod \"416ae4bf-c10a-4ae7-b5a6-8fdeec864c29\" (UID: \"416ae4bf-c10a-4ae7-b5a6-8fdeec864c29\") " May 21 11:48:49.754102 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:49.754049 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/416ae4bf-c10a-4ae7-b5a6-8fdeec864c29-kube-api-access-2mwzk" (OuterVolumeSpecName: "kube-api-access-2mwzk") pod "416ae4bf-c10a-4ae7-b5a6-8fdeec864c29" (UID: "416ae4bf-c10a-4ae7-b5a6-8fdeec864c29"). InnerVolumeSpecName "kube-api-access-2mwzk". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 21 11:48:49.849346 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:49.849302 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2mwzk\" (UniqueName: \"kubernetes.io/projected/416ae4bf-c10a-4ae7-b5a6-8fdeec864c29-kube-api-access-2mwzk\") on node \"ip-10-0-139-213.ec2.internal\" DevicePath \"\"" May 21 11:48:50.405847 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:50.405806 2564 generic.go:358] "Generic (PLEG): container finished" podID="416ae4bf-c10a-4ae7-b5a6-8fdeec864c29" containerID="86a481e9f533480ae36459cf471380a4a5eab8b72a0672bde96d4f7a53460dff" exitCode=0 May 21 11:48:50.405847 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:50.405848 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d4c56f87b-fxl25" event={"ID":"416ae4bf-c10a-4ae7-b5a6-8fdeec864c29","Type":"ContainerDied","Data":"86a481e9f533480ae36459cf471380a4a5eab8b72a0672bde96d4f7a53460dff"} May 21 11:48:50.406370 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:50.405870 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d4c56f87b-fxl25" event={"ID":"416ae4bf-c10a-4ae7-b5a6-8fdeec864c29","Type":"ContainerDied","Data":"142f3d6ca6b131bda4772164471a440c6ea80e7e3881dcc5ca7cec3475e1099a"} May 21 11:48:50.406370 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:50.405885 2564 scope.go:117] "RemoveContainer" containerID="86a481e9f533480ae36459cf471380a4a5eab8b72a0672bde96d4f7a53460dff" May 21 11:48:50.406370 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:50.405891 2564 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d4c56f87b-fxl25" May 21 11:48:50.414673 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:50.414655 2564 scope.go:117] "RemoveContainer" containerID="86a481e9f533480ae36459cf471380a4a5eab8b72a0672bde96d4f7a53460dff" May 21 11:48:50.414929 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:48:50.414909 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86a481e9f533480ae36459cf471380a4a5eab8b72a0672bde96d4f7a53460dff\": container with ID starting with 86a481e9f533480ae36459cf471380a4a5eab8b72a0672bde96d4f7a53460dff not found: ID does not exist" containerID="86a481e9f533480ae36459cf471380a4a5eab8b72a0672bde96d4f7a53460dff" May 21 11:48:50.415000 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:50.414939 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86a481e9f533480ae36459cf471380a4a5eab8b72a0672bde96d4f7a53460dff"} err="failed to get container status \"86a481e9f533480ae36459cf471380a4a5eab8b72a0672bde96d4f7a53460dff\": rpc error: code = NotFound desc = could not find container \"86a481e9f533480ae36459cf471380a4a5eab8b72a0672bde96d4f7a53460dff\": container with ID starting with 86a481e9f533480ae36459cf471380a4a5eab8b72a0672bde96d4f7a53460dff not found: ID does not exist" May 21 11:48:50.428598 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:50.428574 2564 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-d4c56f87b-fxl25"] May 21 11:48:50.431248 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:50.431226 2564 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-d4c56f87b-fxl25"] May 21 11:48:51.684459 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:51.684424 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="416ae4bf-c10a-4ae7-b5a6-8fdeec864c29" path="/var/lib/kubelet/pods/416ae4bf-c10a-4ae7-b5a6-8fdeec864c29/volumes" May 21 11:48:53.426498 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:53.426459 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-79c6b74fcf-rs9b5"] May 21 11:48:53.427014 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:53.426994 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="416ae4bf-c10a-4ae7-b5a6-8fdeec864c29" containerName="manager" May 21 11:48:53.427114 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:53.427019 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="416ae4bf-c10a-4ae7-b5a6-8fdeec864c29" containerName="manager" May 21 11:48:53.427114 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:53.427038 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ad47427-3743-4ab5-8b53-5abec3da61b6" containerName="manager" May 21 11:48:53.427114 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:53.427048 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ad47427-3743-4ab5-8b53-5abec3da61b6" containerName="manager" May 21 11:48:53.427269 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:53.427185 2564 memory_manager.go:356] "RemoveStaleState removing state" podUID="416ae4bf-c10a-4ae7-b5a6-8fdeec864c29" containerName="manager" May 21 11:48:53.427269 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:53.427201 2564 memory_manager.go:356] "RemoveStaleState removing state" podUID="2ad47427-3743-4ab5-8b53-5abec3da61b6" containerName="manager" May 21 11:48:53.432326 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:53.432304 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79c6b74fcf-rs9b5" May 21 11:48:53.439141 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:53.439109 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-79c6b74fcf-rs9b5"] May 21 11:48:53.481247 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:53.481211 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c59j5\" (UniqueName: \"kubernetes.io/projected/8b74d820-2c20-4991-b644-219c36189fec-kube-api-access-c59j5\") pod \"maas-controller-79c6b74fcf-rs9b5\" (UID: \"8b74d820-2c20-4991-b644-219c36189fec\") " pod="opendatahub/maas-controller-79c6b74fcf-rs9b5" May 21 11:48:53.582637 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:53.582597 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-c59j5\" (UniqueName: \"kubernetes.io/projected/8b74d820-2c20-4991-b644-219c36189fec-kube-api-access-c59j5\") pod \"maas-controller-79c6b74fcf-rs9b5\" (UID: \"8b74d820-2c20-4991-b644-219c36189fec\") " pod="opendatahub/maas-controller-79c6b74fcf-rs9b5" May 21 11:48:53.591459 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:53.591427 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c59j5\" (UniqueName: \"kubernetes.io/projected/8b74d820-2c20-4991-b644-219c36189fec-kube-api-access-c59j5\") pod \"maas-controller-79c6b74fcf-rs9b5\" (UID: \"8b74d820-2c20-4991-b644-219c36189fec\") " pod="opendatahub/maas-controller-79c6b74fcf-rs9b5" May 21 11:48:53.745782 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:53.745685 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79c6b74fcf-rs9b5" May 21 11:48:53.882995 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:53.882904 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-79c6b74fcf-rs9b5"] May 21 11:48:53.885161 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:48:53.885130 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b74d820_2c20_4991_b644_219c36189fec.slice/crio-b32db0dbff5d7559865a21ad156701f5767e9381381c07ced6ee7d3fad828060 WatchSource:0}: Error finding container b32db0dbff5d7559865a21ad156701f5767e9381381c07ced6ee7d3fad828060: Status 404 returned error can't find the container with id b32db0dbff5d7559865a21ad156701f5767e9381381c07ced6ee7d3fad828060 May 21 11:48:54.421888 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:54.421850 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79c6b74fcf-rs9b5" event={"ID":"8b74d820-2c20-4991-b644-219c36189fec","Type":"ContainerStarted","Data":"b32db0dbff5d7559865a21ad156701f5767e9381381c07ced6ee7d3fad828060"} May 21 11:48:55.145302 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:55.145261 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-6696777cfc-6pm4j"] May 21 11:48:55.147703 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:55.147678 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-6696777cfc-6pm4j" May 21 11:48:55.150776 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:55.150758 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-88bvc\"" May 21 11:48:55.151239 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:55.151223 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" May 21 11:48:55.159020 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:55.158997 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-6696777cfc-6pm4j"] May 21 11:48:55.196664 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:55.196633 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c7fe35d-142f-423a-b5b6-2420db17fbe9-maas-api-tls\") pod \"maas-api-6696777cfc-6pm4j\" (UID: \"1c7fe35d-142f-423a-b5b6-2420db17fbe9\") " pod="opendatahub/maas-api-6696777cfc-6pm4j" May 21 11:48:55.196808 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:55.196681 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9bdq\" (UniqueName: \"kubernetes.io/projected/1c7fe35d-142f-423a-b5b6-2420db17fbe9-kube-api-access-h9bdq\") pod \"maas-api-6696777cfc-6pm4j\" (UID: \"1c7fe35d-142f-423a-b5b6-2420db17fbe9\") " pod="opendatahub/maas-api-6696777cfc-6pm4j" May 21 11:48:55.297587 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:55.297555 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c7fe35d-142f-423a-b5b6-2420db17fbe9-maas-api-tls\") pod \"maas-api-6696777cfc-6pm4j\" (UID: \"1c7fe35d-142f-423a-b5b6-2420db17fbe9\") " pod="opendatahub/maas-api-6696777cfc-6pm4j" May 21 11:48:55.297587 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:55.297594 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-h9bdq\" (UniqueName: \"kubernetes.io/projected/1c7fe35d-142f-423a-b5b6-2420db17fbe9-kube-api-access-h9bdq\") pod \"maas-api-6696777cfc-6pm4j\" (UID: \"1c7fe35d-142f-423a-b5b6-2420db17fbe9\") " pod="opendatahub/maas-api-6696777cfc-6pm4j" May 21 11:48:55.300335 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:55.300310 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c7fe35d-142f-423a-b5b6-2420db17fbe9-maas-api-tls\") pod \"maas-api-6696777cfc-6pm4j\" (UID: \"1c7fe35d-142f-423a-b5b6-2420db17fbe9\") " pod="opendatahub/maas-api-6696777cfc-6pm4j" May 21 11:48:55.307983 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:55.307957 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9bdq\" (UniqueName: \"kubernetes.io/projected/1c7fe35d-142f-423a-b5b6-2420db17fbe9-kube-api-access-h9bdq\") pod \"maas-api-6696777cfc-6pm4j\" (UID: \"1c7fe35d-142f-423a-b5b6-2420db17fbe9\") " pod="opendatahub/maas-api-6696777cfc-6pm4j" May 21 11:48:55.434792 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:55.434709 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79c6b74fcf-rs9b5" event={"ID":"8b74d820-2c20-4991-b644-219c36189fec","Type":"ContainerStarted","Data":"257931ebee7857e766eb084dc649e96fae0f98df77289f7746ba3112cdcc8234"} May 21 11:48:55.434971 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:55.434952 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-79c6b74fcf-rs9b5" May 21 11:48:55.455791 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:55.455748 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-79c6b74fcf-rs9b5" podStartSLOduration=1.79595693 podStartE2EDuration="2.45573491s" podCreationTimestamp="2026-05-21 11:48:53 +0000 UTC" firstStartedPulling="2026-05-21 11:48:53.886527668 +0000 UTC m=+1088.781811191" lastFinishedPulling="2026-05-21 11:48:54.546305629 +0000 UTC m=+1089.441589171" observedRunningTime="2026-05-21 11:48:55.454129225 +0000 UTC m=+1090.349412772" watchObservedRunningTime="2026-05-21 11:48:55.45573491 +0000 UTC m=+1090.351018454" May 21 11:48:55.459942 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:55.459919 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-6696777cfc-6pm4j" May 21 11:48:55.499650 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:55.499613 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58b6c8fdc7-f2wp8"] May 21 11:48:55.502559 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:55.502537 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58b6c8fdc7-f2wp8" May 21 11:48:55.505764 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:55.505738 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" May 21 11:48:55.505960 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:55.505938 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-wq64x\"" May 21 11:48:55.516204 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:55.516161 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58b6c8fdc7-f2wp8"] May 21 11:48:55.598586 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:55.598486 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-6696777cfc-6pm4j"] May 21 11:48:55.600689 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:55.600662 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8jt8\" (UniqueName: \"kubernetes.io/projected/74a00797-92ed-4a1e-83e6-4ec7f69eaec1-kube-api-access-x8jt8\") pod \"payload-processing-58b6c8fdc7-f2wp8\" (UID: \"74a00797-92ed-4a1e-83e6-4ec7f69eaec1\") " pod="openshift-ingress/payload-processing-58b6c8fdc7-f2wp8" May 21 11:48:55.600798 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:48:55.600678 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c7fe35d_142f_423a_b5b6_2420db17fbe9.slice/crio-0b02f52b3b433f48c1c59488a4415e0a508669f2c5ce06a25fd2689fe0e4f31f WatchSource:0}: Error finding container 0b02f52b3b433f48c1c59488a4415e0a508669f2c5ce06a25fd2689fe0e4f31f: Status 404 returned error can't find the container with id 0b02f52b3b433f48c1c59488a4415e0a508669f2c5ce06a25fd2689fe0e4f31f May 21 11:48:55.701375 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:55.701294 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-x8jt8\" (UniqueName: \"kubernetes.io/projected/74a00797-92ed-4a1e-83e6-4ec7f69eaec1-kube-api-access-x8jt8\") pod \"payload-processing-58b6c8fdc7-f2wp8\" (UID: \"74a00797-92ed-4a1e-83e6-4ec7f69eaec1\") " pod="openshift-ingress/payload-processing-58b6c8fdc7-f2wp8" May 21 11:48:55.710428 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:55.710394 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8jt8\" (UniqueName: \"kubernetes.io/projected/74a00797-92ed-4a1e-83e6-4ec7f69eaec1-kube-api-access-x8jt8\") pod \"payload-processing-58b6c8fdc7-f2wp8\" (UID: \"74a00797-92ed-4a1e-83e6-4ec7f69eaec1\") " pod="openshift-ingress/payload-processing-58b6c8fdc7-f2wp8" May 21 11:48:55.821297 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:55.821261 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58b6c8fdc7-f2wp8" May 21 11:48:55.949695 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:55.949663 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58b6c8fdc7-f2wp8"] May 21 11:48:55.952791 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:48:55.952731 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74a00797_92ed_4a1e_83e6_4ec7f69eaec1.slice/crio-774ce3302364218fde7c3061f436304eadc77d83748182a89c4a4c185788542d WatchSource:0}: Error finding container 774ce3302364218fde7c3061f436304eadc77d83748182a89c4a4c185788542d: Status 404 returned error can't find the container with id 774ce3302364218fde7c3061f436304eadc77d83748182a89c4a4c185788542d May 21 11:48:56.439261 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:56.439225 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-6696777cfc-6pm4j" event={"ID":"1c7fe35d-142f-423a-b5b6-2420db17fbe9","Type":"ContainerStarted","Data":"0b02f52b3b433f48c1c59488a4415e0a508669f2c5ce06a25fd2689fe0e4f31f"} May 21 11:48:56.440277 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:56.440253 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58b6c8fdc7-f2wp8" event={"ID":"74a00797-92ed-4a1e-83e6-4ec7f69eaec1","Type":"ContainerStarted","Data":"774ce3302364218fde7c3061f436304eadc77d83748182a89c4a4c185788542d"} May 21 11:48:57.445763 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:57.445726 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-6696777cfc-6pm4j" event={"ID":"1c7fe35d-142f-423a-b5b6-2420db17fbe9","Type":"ContainerStarted","Data":"dbc9b16fa3f1a59c7c2b864239a7485617461273e379fe2ba39d95844c11a630"} May 21 11:48:57.446194 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:57.445870 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-6696777cfc-6pm4j" May 21 11:48:57.465442 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:57.465384 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-api-6696777cfc-6pm4j" podStartSLOduration=0.820432312 podStartE2EDuration="2.465368285s" podCreationTimestamp="2026-05-21 11:48:55 +0000 UTC" firstStartedPulling="2026-05-21 11:48:55.602049525 +0000 UTC m=+1090.497333051" lastFinishedPulling="2026-05-21 11:48:57.246985499 +0000 UTC m=+1092.142269024" observedRunningTime="2026-05-21 11:48:57.463526314 +0000 UTC m=+1092.358809858" watchObservedRunningTime="2026-05-21 11:48:57.465368285 +0000 UTC m=+1092.360651830" May 21 11:48:58.450727 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:58.450695 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58b6c8fdc7-f2wp8" event={"ID":"74a00797-92ed-4a1e-83e6-4ec7f69eaec1","Type":"ContainerStarted","Data":"361db28cab850701f25ee0202a53e2f90d46b8c8ba479c3d24878fed9302e8b0"} May 21 11:48:58.468321 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:58.468271 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58b6c8fdc7-f2wp8" podStartSLOduration=1.100757596 podStartE2EDuration="3.468254537s" podCreationTimestamp="2026-05-21 11:48:55 +0000 UTC" firstStartedPulling="2026-05-21 11:48:55.954574679 +0000 UTC m=+1090.849858205" lastFinishedPulling="2026-05-21 11:48:58.322071617 +0000 UTC m=+1093.217355146" observedRunningTime="2026-05-21 11:48:58.46669906 +0000 UTC m=+1093.361982616" watchObservedRunningTime="2026-05-21 11:48:58.468254537 +0000 UTC m=+1093.363538136" May 21 11:48:59.454869 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:48:59.454827 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58b6c8fdc7-f2wp8" May 21 11:49:03.456464 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:49:03.456435 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-6696777cfc-6pm4j" May 21 11:49:06.446388 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:49:06.446353 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-79c6b74fcf-rs9b5" May 21 11:49:06.493473 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:49:06.493439 2564 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7f6fcccfc8-7bzvn"] May 21 11:49:06.493689 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:49:06.493661 2564 kuberuntime_container.go:864] "Killing container with a grace period" pod="opendatahub/maas-controller-7f6fcccfc8-7bzvn" podUID="55a65d66-020b-4e51-b189-f6dc806cc77c" containerName="manager" containerID="cri-o://27ac2f932a99c4acf0b7036d9cf9f4459e63cbba4434322265666bfa3f1428da" gracePeriod=30 May 21 11:49:06.741181 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:49:06.741162 2564 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f6fcccfc8-7bzvn" May 21 11:49:06.800802 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:49:06.800768 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p4n4m\" (UniqueName: \"kubernetes.io/projected/55a65d66-020b-4e51-b189-f6dc806cc77c-kube-api-access-p4n4m\") pod \"55a65d66-020b-4e51-b189-f6dc806cc77c\" (UID: \"55a65d66-020b-4e51-b189-f6dc806cc77c\") " May 21 11:49:06.803769 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:49:06.803724 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55a65d66-020b-4e51-b189-f6dc806cc77c-kube-api-access-p4n4m" (OuterVolumeSpecName: "kube-api-access-p4n4m") pod "55a65d66-020b-4e51-b189-f6dc806cc77c" (UID: "55a65d66-020b-4e51-b189-f6dc806cc77c"). InnerVolumeSpecName "kube-api-access-p4n4m". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 21 11:49:06.902314 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:49:06.902280 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p4n4m\" (UniqueName: \"kubernetes.io/projected/55a65d66-020b-4e51-b189-f6dc806cc77c-kube-api-access-p4n4m\") on node \"ip-10-0-139-213.ec2.internal\" DevicePath \"\"" May 21 11:49:07.485806 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:49:07.485766 2564 generic.go:358] "Generic (PLEG): container finished" podID="55a65d66-020b-4e51-b189-f6dc806cc77c" containerID="27ac2f932a99c4acf0b7036d9cf9f4459e63cbba4434322265666bfa3f1428da" exitCode=0 May 21 11:49:07.486251 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:49:07.485839 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f6fcccfc8-7bzvn" event={"ID":"55a65d66-020b-4e51-b189-f6dc806cc77c","Type":"ContainerDied","Data":"27ac2f932a99c4acf0b7036d9cf9f4459e63cbba4434322265666bfa3f1428da"} May 21 11:49:07.486251 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:49:07.485847 2564 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f6fcccfc8-7bzvn" May 21 11:49:07.486251 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:49:07.485869 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f6fcccfc8-7bzvn" event={"ID":"55a65d66-020b-4e51-b189-f6dc806cc77c","Type":"ContainerDied","Data":"c2fa53ed1fac18ba8ab5ba322a54186cb1862c98977add00ba29ba0c80a7867b"} May 21 11:49:07.486251 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:49:07.485886 2564 scope.go:117] "RemoveContainer" containerID="27ac2f932a99c4acf0b7036d9cf9f4459e63cbba4434322265666bfa3f1428da" May 21 11:49:07.495587 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:49:07.495481 2564 scope.go:117] "RemoveContainer" containerID="27ac2f932a99c4acf0b7036d9cf9f4459e63cbba4434322265666bfa3f1428da" May 21 11:49:07.495764 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:49:07.495738 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27ac2f932a99c4acf0b7036d9cf9f4459e63cbba4434322265666bfa3f1428da\": container with ID starting with 27ac2f932a99c4acf0b7036d9cf9f4459e63cbba4434322265666bfa3f1428da not found: ID does not exist" containerID="27ac2f932a99c4acf0b7036d9cf9f4459e63cbba4434322265666bfa3f1428da" May 21 11:49:07.495862 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:49:07.495770 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27ac2f932a99c4acf0b7036d9cf9f4459e63cbba4434322265666bfa3f1428da"} err="failed to get container status \"27ac2f932a99c4acf0b7036d9cf9f4459e63cbba4434322265666bfa3f1428da\": rpc error: code = NotFound desc = could not find container \"27ac2f932a99c4acf0b7036d9cf9f4459e63cbba4434322265666bfa3f1428da\": container with ID starting with 27ac2f932a99c4acf0b7036d9cf9f4459e63cbba4434322265666bfa3f1428da not found: ID does not exist" May 21 11:49:07.506961 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:49:07.506935 2564 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7f6fcccfc8-7bzvn"] May 21 11:49:07.508757 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:49:07.508739 2564 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7f6fcccfc8-7bzvn"] May 21 11:49:07.684361 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:49:07.684333 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55a65d66-020b-4e51-b189-f6dc806cc77c" path="/var/lib/kubelet/pods/55a65d66-020b-4e51-b189-f6dc806cc77c/volumes" May 21 11:49:10.461020 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:49:10.460990 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58b6c8fdc7-f2wp8" May 21 11:49:25.089616 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:49:25.086587 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-8687598f7d-6z85m"] May 21 11:49:25.089616 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:49:25.087532 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55a65d66-020b-4e51-b189-f6dc806cc77c" containerName="manager" May 21 11:49:25.089616 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:49:25.087555 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="55a65d66-020b-4e51-b189-f6dc806cc77c" containerName="manager" May 21 11:49:25.089616 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:49:25.087754 2564 memory_manager.go:356] "RemoveStaleState removing state" podUID="55a65d66-020b-4e51-b189-f6dc806cc77c" containerName="manager" May 21 11:49:25.093115 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:49:25.093069 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-8687598f7d-6z85m" May 21 11:49:25.098484 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:49:25.098459 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-8687598f7d-6z85m"] May 21 11:49:25.272772 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:49:25.272730 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b1b3588-0b02-4a55-907a-64b9513bf1df-maas-api-tls\") pod \"maas-api-8687598f7d-6z85m\" (UID: \"9b1b3588-0b02-4a55-907a-64b9513bf1df\") " pod="opendatahub/maas-api-8687598f7d-6z85m" May 21 11:49:25.272966 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:49:25.272810 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kld9v\" (UniqueName: \"kubernetes.io/projected/9b1b3588-0b02-4a55-907a-64b9513bf1df-kube-api-access-kld9v\") pod \"maas-api-8687598f7d-6z85m\" (UID: \"9b1b3588-0b02-4a55-907a-64b9513bf1df\") " pod="opendatahub/maas-api-8687598f7d-6z85m" May 21 11:49:25.374237 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:49:25.374135 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b1b3588-0b02-4a55-907a-64b9513bf1df-maas-api-tls\") pod \"maas-api-8687598f7d-6z85m\" (UID: \"9b1b3588-0b02-4a55-907a-64b9513bf1df\") " pod="opendatahub/maas-api-8687598f7d-6z85m" May 21 11:49:25.374237 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:49:25.374214 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kld9v\" (UniqueName: \"kubernetes.io/projected/9b1b3588-0b02-4a55-907a-64b9513bf1df-kube-api-access-kld9v\") pod \"maas-api-8687598f7d-6z85m\" (UID: \"9b1b3588-0b02-4a55-907a-64b9513bf1df\") " pod="opendatahub/maas-api-8687598f7d-6z85m" May 21 11:49:25.376804 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:49:25.376784 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b1b3588-0b02-4a55-907a-64b9513bf1df-maas-api-tls\") pod \"maas-api-8687598f7d-6z85m\" (UID: \"9b1b3588-0b02-4a55-907a-64b9513bf1df\") " pod="opendatahub/maas-api-8687598f7d-6z85m" May 21 11:49:25.383054 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:49:25.383030 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kld9v\" (UniqueName: \"kubernetes.io/projected/9b1b3588-0b02-4a55-907a-64b9513bf1df-kube-api-access-kld9v\") pod \"maas-api-8687598f7d-6z85m\" (UID: \"9b1b3588-0b02-4a55-907a-64b9513bf1df\") " pod="opendatahub/maas-api-8687598f7d-6z85m" May 21 11:49:25.406831 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:49:25.406809 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-8687598f7d-6z85m" May 21 11:49:25.541009 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:49:25.540985 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-8687598f7d-6z85m"] May 21 11:49:25.544486 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:49:25.544453 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b1b3588_0b02_4a55_907a_64b9513bf1df.slice/crio-bbb6f022cacaf11b4029e230155a8d0f3dc977626567118b954ff702bf6466e9 WatchSource:0}: Error finding container bbb6f022cacaf11b4029e230155a8d0f3dc977626567118b954ff702bf6466e9: Status 404 returned error can't find the container with id bbb6f022cacaf11b4029e230155a8d0f3dc977626567118b954ff702bf6466e9 May 21 11:49:25.553724 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:49:25.553696 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-8687598f7d-6z85m" event={"ID":"9b1b3588-0b02-4a55-907a-64b9513bf1df","Type":"ContainerStarted","Data":"bbb6f022cacaf11b4029e230155a8d0f3dc977626567118b954ff702bf6466e9"} May 21 11:49:27.562120 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:49:27.562056 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-8687598f7d-6z85m" event={"ID":"9b1b3588-0b02-4a55-907a-64b9513bf1df","Type":"ContainerStarted","Data":"12f39f8a79c9475ede0f55170f551731d0698e59ec402cf0af9fa3c84cd14bde"} May 21 11:49:27.562513 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:49:27.562143 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-8687598f7d-6z85m" May 21 11:49:27.616264 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:49:27.616214 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-api-8687598f7d-6z85m" podStartSLOduration=1.165385317 podStartE2EDuration="2.616199478s" podCreationTimestamp="2026-05-21 11:49:25 +0000 UTC" firstStartedPulling="2026-05-21 11:49:25.54595383 +0000 UTC m=+1120.441237368" lastFinishedPulling="2026-05-21 11:49:26.996767994 +0000 UTC m=+1121.892051529" observedRunningTime="2026-05-21 11:49:27.614522509 +0000 UTC m=+1122.509806054" watchObservedRunningTime="2026-05-21 11:49:27.616199478 +0000 UTC m=+1122.511483022" May 21 11:49:33.570892 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:49:33.570861 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-8687598f7d-6z85m" May 21 11:49:33.622327 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:49:33.622296 2564 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-6696777cfc-6pm4j"] May 21 11:49:33.622586 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:49:33.622536 2564 kuberuntime_container.go:864] "Killing container with a grace period" pod="opendatahub/maas-api-6696777cfc-6pm4j" podUID="1c7fe35d-142f-423a-b5b6-2420db17fbe9" containerName="maas-api" containerID="cri-o://dbc9b16fa3f1a59c7c2b864239a7485617461273e379fe2ba39d95844c11a630" gracePeriod=30 May 21 11:49:33.860571 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:49:33.860548 2564 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-6696777cfc-6pm4j" May 21 11:49:33.951806 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:49:33.951775 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c7fe35d-142f-423a-b5b6-2420db17fbe9-maas-api-tls\") pod \"1c7fe35d-142f-423a-b5b6-2420db17fbe9\" (UID: \"1c7fe35d-142f-423a-b5b6-2420db17fbe9\") " May 21 11:49:33.951970 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:49:33.951836 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h9bdq\" (UniqueName: \"kubernetes.io/projected/1c7fe35d-142f-423a-b5b6-2420db17fbe9-kube-api-access-h9bdq\") pod \"1c7fe35d-142f-423a-b5b6-2420db17fbe9\" (UID: \"1c7fe35d-142f-423a-b5b6-2420db17fbe9\") " May 21 11:49:33.954122 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:49:33.954093 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c7fe35d-142f-423a-b5b6-2420db17fbe9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1c7fe35d-142f-423a-b5b6-2420db17fbe9" (UID: "1c7fe35d-142f-423a-b5b6-2420db17fbe9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 21 11:49:33.954253 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:49:33.954146 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c7fe35d-142f-423a-b5b6-2420db17fbe9-kube-api-access-h9bdq" (OuterVolumeSpecName: "kube-api-access-h9bdq") pod "1c7fe35d-142f-423a-b5b6-2420db17fbe9" (UID: "1c7fe35d-142f-423a-b5b6-2420db17fbe9"). InnerVolumeSpecName "kube-api-access-h9bdq". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 21 11:49:34.052573 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:49:34.052544 2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c7fe35d-142f-423a-b5b6-2420db17fbe9-maas-api-tls\") on node \"ip-10-0-139-213.ec2.internal\" DevicePath \"\"" May 21 11:49:34.052573 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:49:34.052570 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h9bdq\" (UniqueName: \"kubernetes.io/projected/1c7fe35d-142f-423a-b5b6-2420db17fbe9-kube-api-access-h9bdq\") on node \"ip-10-0-139-213.ec2.internal\" DevicePath \"\"" May 21 11:49:34.587376 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:49:34.587339 2564 generic.go:358] "Generic (PLEG): container finished" podID="1c7fe35d-142f-423a-b5b6-2420db17fbe9" containerID="dbc9b16fa3f1a59c7c2b864239a7485617461273e379fe2ba39d95844c11a630" exitCode=0 May 21 11:49:34.587857 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:49:34.587402 2564 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-6696777cfc-6pm4j" May 21 11:49:34.587857 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:49:34.587426 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-6696777cfc-6pm4j" event={"ID":"1c7fe35d-142f-423a-b5b6-2420db17fbe9","Type":"ContainerDied","Data":"dbc9b16fa3f1a59c7c2b864239a7485617461273e379fe2ba39d95844c11a630"} May 21 11:49:34.587857 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:49:34.587464 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-6696777cfc-6pm4j" event={"ID":"1c7fe35d-142f-423a-b5b6-2420db17fbe9","Type":"ContainerDied","Data":"0b02f52b3b433f48c1c59488a4415e0a508669f2c5ce06a25fd2689fe0e4f31f"} May 21 11:49:34.587857 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:49:34.587487 2564 scope.go:117] "RemoveContainer" containerID="dbc9b16fa3f1a59c7c2b864239a7485617461273e379fe2ba39d95844c11a630" May 21 11:49:34.596011 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:49:34.595995 2564 scope.go:117] "RemoveContainer" containerID="dbc9b16fa3f1a59c7c2b864239a7485617461273e379fe2ba39d95844c11a630" May 21 11:49:34.596321 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:49:34.596300 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dbc9b16fa3f1a59c7c2b864239a7485617461273e379fe2ba39d95844c11a630\": container with ID starting with dbc9b16fa3f1a59c7c2b864239a7485617461273e379fe2ba39d95844c11a630 not found: ID does not exist" containerID="dbc9b16fa3f1a59c7c2b864239a7485617461273e379fe2ba39d95844c11a630" May 21 11:49:34.596382 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:49:34.596328 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbc9b16fa3f1a59c7c2b864239a7485617461273e379fe2ba39d95844c11a630"} err="failed to get container status \"dbc9b16fa3f1a59c7c2b864239a7485617461273e379fe2ba39d95844c11a630\": rpc error: code = NotFound desc = could not find container \"dbc9b16fa3f1a59c7c2b864239a7485617461273e379fe2ba39d95844c11a630\": container with ID starting with dbc9b16fa3f1a59c7c2b864239a7485617461273e379fe2ba39d95844c11a630 not found: ID does not exist" May 21 11:49:34.610122 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:49:34.610099 2564 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-6696777cfc-6pm4j"] May 21 11:49:34.614334 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:49:34.614313 2564 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-6696777cfc-6pm4j"] May 21 11:49:35.683964 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:49:35.683921 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c7fe35d-142f-423a-b5b6-2420db17fbe9" path="/var/lib/kubelet/pods/1c7fe35d-142f-423a-b5b6-2420db17fbe9/volumes" May 21 11:50:02.631454 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:02.631372 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-wrv26"] May 21 11:50:02.631846 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:02.631744 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c7fe35d-142f-423a-b5b6-2420db17fbe9" containerName="maas-api" May 21 11:50:02.631846 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:02.631755 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c7fe35d-142f-423a-b5b6-2420db17fbe9" containerName="maas-api" May 21 11:50:02.631846 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:02.631836 2564 memory_manager.go:356] "RemoveStaleState removing state" podUID="1c7fe35d-142f-423a-b5b6-2420db17fbe9" containerName="maas-api" May 21 11:50:02.634912 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:02.634896 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-wrv26" May 21 11:50:02.638245 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:02.638217 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" May 21 11:50:02.639716 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:02.639697 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" May 21 11:50:02.639816 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:02.639718 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-jsq4f\"" May 21 11:50:02.639816 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:02.639740 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" May 21 11:50:02.648469 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:02.648446 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-wrv26"] May 21 11:50:02.720675 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:02.720642 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vl8gv\" (UniqueName: \"kubernetes.io/projected/450a0d73-c37b-490e-9333-c311f927b961-kube-api-access-vl8gv\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-wrv26\" (UID: \"450a0d73-c37b-490e-9333-c311f927b961\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-wrv26" May 21 11:50:02.720858 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:02.720698 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/450a0d73-c37b-490e-9333-c311f927b961-model-cache\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-wrv26\" (UID: \"450a0d73-c37b-490e-9333-c311f927b961\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-wrv26" May 21 11:50:02.720858 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:02.720841 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/450a0d73-c37b-490e-9333-c311f927b961-home\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-wrv26\" (UID: \"450a0d73-c37b-490e-9333-c311f927b961\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-wrv26" May 21 11:50:02.721028 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:02.721007 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/450a0d73-c37b-490e-9333-c311f927b961-dshm\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-wrv26\" (UID: \"450a0d73-c37b-490e-9333-c311f927b961\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-wrv26" May 21 11:50:02.721161 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:02.721137 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/450a0d73-c37b-490e-9333-c311f927b961-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-wrv26\" (UID: \"450a0d73-c37b-490e-9333-c311f927b961\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-wrv26" May 21 11:50:02.721281 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:02.721254 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/450a0d73-c37b-490e-9333-c311f927b961-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-wrv26\" (UID: \"450a0d73-c37b-490e-9333-c311f927b961\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-wrv26" May 21 11:50:02.821722 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:02.821684 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/450a0d73-c37b-490e-9333-c311f927b961-dshm\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-wrv26\" (UID: \"450a0d73-c37b-490e-9333-c311f927b961\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-wrv26" May 21 11:50:02.821883 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:02.821738 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/450a0d73-c37b-490e-9333-c311f927b961-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-wrv26\" (UID: \"450a0d73-c37b-490e-9333-c311f927b961\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-wrv26" May 21 11:50:02.821883 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:02.821788 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/450a0d73-c37b-490e-9333-c311f927b961-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-wrv26\" (UID: \"450a0d73-c37b-490e-9333-c311f927b961\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-wrv26" May 21 11:50:02.821883 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:02.821815 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vl8gv\" (UniqueName: \"kubernetes.io/projected/450a0d73-c37b-490e-9333-c311f927b961-kube-api-access-vl8gv\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-wrv26\" (UID: \"450a0d73-c37b-490e-9333-c311f927b961\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-wrv26" May 21 11:50:02.821883 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:02.821849 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/450a0d73-c37b-490e-9333-c311f927b961-model-cache\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-wrv26\" (UID: \"450a0d73-c37b-490e-9333-c311f927b961\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-wrv26" May 21 11:50:02.822061 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:02.821890 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/450a0d73-c37b-490e-9333-c311f927b961-home\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-wrv26\" (UID: \"450a0d73-c37b-490e-9333-c311f927b961\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-wrv26" May 21 11:50:02.822199 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:02.822177 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/450a0d73-c37b-490e-9333-c311f927b961-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-wrv26\" (UID: \"450a0d73-c37b-490e-9333-c311f927b961\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-wrv26" May 21 11:50:02.822293 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:02.822269 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/450a0d73-c37b-490e-9333-c311f927b961-home\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-wrv26\" (UID: \"450a0d73-c37b-490e-9333-c311f927b961\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-wrv26" May 21 11:50:02.822397 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:02.822326 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/450a0d73-c37b-490e-9333-c311f927b961-model-cache\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-wrv26\" (UID: \"450a0d73-c37b-490e-9333-c311f927b961\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-wrv26" May 21 11:50:02.824197 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:02.824179 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/450a0d73-c37b-490e-9333-c311f927b961-dshm\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-wrv26\" (UID: \"450a0d73-c37b-490e-9333-c311f927b961\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-wrv26" May 21 11:50:02.824481 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:02.824463 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/450a0d73-c37b-490e-9333-c311f927b961-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-wrv26\" (UID: \"450a0d73-c37b-490e-9333-c311f927b961\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-wrv26" May 21 11:50:02.832415 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:02.832396 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vl8gv\" (UniqueName: \"kubernetes.io/projected/450a0d73-c37b-490e-9333-c311f927b961-kube-api-access-vl8gv\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-wrv26\" (UID: \"450a0d73-c37b-490e-9333-c311f927b961\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-wrv26" May 21 11:50:02.945311 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:02.945226 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-wrv26" May 21 11:50:03.080645 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:03.080620 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-wrv26"] May 21 11:50:03.082535 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:50:03.082510 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod450a0d73_c37b_490e_9333_c311f927b961.slice/crio-29fdaae1eabfe8042541a58b70e7e8f8208fda8f71cf20ed114732ab32822c8a WatchSource:0}: Error finding container 29fdaae1eabfe8042541a58b70e7e8f8208fda8f71cf20ed114732ab32822c8a: Status 404 returned error can't find the container with id 29fdaae1eabfe8042541a58b70e7e8f8208fda8f71cf20ed114732ab32822c8a May 21 11:50:03.689635 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:03.689595 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-wrv26" event={"ID":"450a0d73-c37b-490e-9333-c311f927b961","Type":"ContainerStarted","Data":"29fdaae1eabfe8042541a58b70e7e8f8208fda8f71cf20ed114732ab32822c8a"} May 21 11:50:09.714189 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:09.714150 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-wrv26" event={"ID":"450a0d73-c37b-490e-9333-c311f927b961","Type":"ContainerStarted","Data":"b955bdc95ff4fa35697da3f681be4d0dd3607ff7aff86bcfa604ad401d6c32ef"} May 21 11:50:14.594630 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:14.594599 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmfwzw"] May 21 11:50:14.599490 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:14.599470 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmfwzw" May 21 11:50:14.602937 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:14.602917 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" May 21 11:50:14.609694 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:14.609669 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmfwzw"] May 21 11:50:14.622665 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:14.622641 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnprz\" (UniqueName: \"kubernetes.io/projected/7eddd6e5-1cf0-4694-bdc9-ab88d755537a-kube-api-access-pnprz\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmfwzw\" (UID: \"7eddd6e5-1cf0-4694-bdc9-ab88d755537a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmfwzw" May 21 11:50:14.622780 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:14.622687 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7eddd6e5-1cf0-4694-bdc9-ab88d755537a-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmfwzw\" (UID: \"7eddd6e5-1cf0-4694-bdc9-ab88d755537a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmfwzw" May 21 11:50:14.622780 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:14.622727 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7eddd6e5-1cf0-4694-bdc9-ab88d755537a-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmfwzw\" (UID: \"7eddd6e5-1cf0-4694-bdc9-ab88d755537a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmfwzw" May 21 11:50:14.622909 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:14.622850 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7eddd6e5-1cf0-4694-bdc9-ab88d755537a-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmfwzw\" (UID: \"7eddd6e5-1cf0-4694-bdc9-ab88d755537a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmfwzw" May 21 11:50:14.622909 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:14.622891 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7eddd6e5-1cf0-4694-bdc9-ab88d755537a-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmfwzw\" (UID: \"7eddd6e5-1cf0-4694-bdc9-ab88d755537a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmfwzw" May 21 11:50:14.622996 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:14.622922 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7eddd6e5-1cf0-4694-bdc9-ab88d755537a-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmfwzw\" (UID: \"7eddd6e5-1cf0-4694-bdc9-ab88d755537a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmfwzw" May 21 11:50:14.724469 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:14.724423 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7eddd6e5-1cf0-4694-bdc9-ab88d755537a-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmfwzw\" (UID: \"7eddd6e5-1cf0-4694-bdc9-ab88d755537a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmfwzw" May 21 11:50:14.724469 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:14.724474 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7eddd6e5-1cf0-4694-bdc9-ab88d755537a-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmfwzw\" (UID: \"7eddd6e5-1cf0-4694-bdc9-ab88d755537a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmfwzw" May 21 11:50:14.724724 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:14.724499 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7eddd6e5-1cf0-4694-bdc9-ab88d755537a-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmfwzw\" (UID: \"7eddd6e5-1cf0-4694-bdc9-ab88d755537a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmfwzw" May 21 11:50:14.724724 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:14.724537 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pnprz\" (UniqueName: \"kubernetes.io/projected/7eddd6e5-1cf0-4694-bdc9-ab88d755537a-kube-api-access-pnprz\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmfwzw\" (UID: \"7eddd6e5-1cf0-4694-bdc9-ab88d755537a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmfwzw" May 21 11:50:14.724912 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:14.724881 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7eddd6e5-1cf0-4694-bdc9-ab88d755537a-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmfwzw\" (UID: \"7eddd6e5-1cf0-4694-bdc9-ab88d755537a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmfwzw" May 21 11:50:14.725069 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:14.724928 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7eddd6e5-1cf0-4694-bdc9-ab88d755537a-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmfwzw\" (UID: \"7eddd6e5-1cf0-4694-bdc9-ab88d755537a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmfwzw" May 21 11:50:14.725069 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:14.724881 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7eddd6e5-1cf0-4694-bdc9-ab88d755537a-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmfwzw\" (UID: \"7eddd6e5-1cf0-4694-bdc9-ab88d755537a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmfwzw" May 21 11:50:14.725222 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:14.725148 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7eddd6e5-1cf0-4694-bdc9-ab88d755537a-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmfwzw\" (UID: \"7eddd6e5-1cf0-4694-bdc9-ab88d755537a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmfwzw" May 21 11:50:14.725448 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:14.725283 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7eddd6e5-1cf0-4694-bdc9-ab88d755537a-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmfwzw\" (UID: \"7eddd6e5-1cf0-4694-bdc9-ab88d755537a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmfwzw" May 21 11:50:14.727178 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:14.727155 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7eddd6e5-1cf0-4694-bdc9-ab88d755537a-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmfwzw\" (UID: \"7eddd6e5-1cf0-4694-bdc9-ab88d755537a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmfwzw" May 21 11:50:14.727319 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:14.727302 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7eddd6e5-1cf0-4694-bdc9-ab88d755537a-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmfwzw\" (UID: \"7eddd6e5-1cf0-4694-bdc9-ab88d755537a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmfwzw" May 21 11:50:14.733977 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:14.733950 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnprz\" (UniqueName: \"kubernetes.io/projected/7eddd6e5-1cf0-4694-bdc9-ab88d755537a-kube-api-access-pnprz\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmfwzw\" (UID: \"7eddd6e5-1cf0-4694-bdc9-ab88d755537a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmfwzw" May 21 11:50:14.910181 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:14.910149 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmfwzw" May 21 11:50:15.057423 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:15.057208 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmfwzw"] May 21 11:50:15.060126 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:50:15.060068 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7eddd6e5_1cf0_4694_bdc9_ab88d755537a.slice/crio-2475405aaf70eca4f1f26ba12da2f8392f8b262063630b49d4980eeeed4f1c04 WatchSource:0}: Error finding container 2475405aaf70eca4f1f26ba12da2f8392f8b262063630b49d4980eeeed4f1c04: Status 404 returned error can't find the container with id 2475405aaf70eca4f1f26ba12da2f8392f8b262063630b49d4980eeeed4f1c04 May 21 11:50:15.738132 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:15.738089 2564 generic.go:358] "Generic (PLEG): container finished" podID="450a0d73-c37b-490e-9333-c311f927b961" containerID="b955bdc95ff4fa35697da3f681be4d0dd3607ff7aff86bcfa604ad401d6c32ef" exitCode=0 May 21 11:50:15.738708 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:15.738103 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-wrv26" event={"ID":"450a0d73-c37b-490e-9333-c311f927b961","Type":"ContainerDied","Data":"b955bdc95ff4fa35697da3f681be4d0dd3607ff7aff86bcfa604ad401d6c32ef"} May 21 11:50:15.739808 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:15.739782 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmfwzw" event={"ID":"7eddd6e5-1cf0-4694-bdc9-ab88d755537a","Type":"ContainerStarted","Data":"412c09f3c11a98531feabfa1a653d8c87b201c4723f2dc97699848736c8df00d"} May 21 11:50:15.739959 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:15.739933 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmfwzw" event={"ID":"7eddd6e5-1cf0-4694-bdc9-ab88d755537a","Type":"ContainerStarted","Data":"2475405aaf70eca4f1f26ba12da2f8392f8b262063630b49d4980eeeed4f1c04"} May 21 11:50:17.751685 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:17.751647 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-wrv26" event={"ID":"450a0d73-c37b-490e-9333-c311f927b961","Type":"ContainerStarted","Data":"3b04bfbcc04f52f53b7981366886822b6baf7c4b6fde7681b753e5145cf195d4"} May 21 11:50:17.752141 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:17.751979 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-wrv26" May 21 11:50:17.789606 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:17.789541 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-wrv26" podStartSLOduration=1.8731709539999999 podStartE2EDuration="15.789527633s" podCreationTimestamp="2026-05-21 11:50:02 +0000 UTC" firstStartedPulling="2026-05-21 11:50:03.091511882 +0000 UTC m=+1157.986795407" lastFinishedPulling="2026-05-21 11:50:17.007868559 +0000 UTC m=+1171.903152086" observedRunningTime="2026-05-21 11:50:17.788546143 +0000 UTC m=+1172.683829688" watchObservedRunningTime="2026-05-21 11:50:17.789527633 +0000 UTC m=+1172.684811177" May 21 11:50:21.769232 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:21.769194 2564 generic.go:358] "Generic (PLEG): container finished" podID="7eddd6e5-1cf0-4694-bdc9-ab88d755537a" containerID="412c09f3c11a98531feabfa1a653d8c87b201c4723f2dc97699848736c8df00d" exitCode=0 May 21 11:50:21.769627 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:21.769264 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmfwzw" event={"ID":"7eddd6e5-1cf0-4694-bdc9-ab88d755537a","Type":"ContainerDied","Data":"412c09f3c11a98531feabfa1a653d8c87b201c4723f2dc97699848736c8df00d"} May 21 11:50:22.775918 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:22.775878 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmfwzw" event={"ID":"7eddd6e5-1cf0-4694-bdc9-ab88d755537a","Type":"ContainerStarted","Data":"a208f9e20bafc2aea416d5961bf21458fc20eacdb87dd9d57685b96455aa61d3"} May 21 11:50:22.776344 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:22.776225 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmfwzw" May 21 11:50:22.804350 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:22.804297 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmfwzw" podStartSLOduration=8.501232225999999 podStartE2EDuration="8.804278656s" podCreationTimestamp="2026-05-21 11:50:14 +0000 UTC" firstStartedPulling="2026-05-21 11:50:21.769970442 +0000 UTC m=+1176.665253969" lastFinishedPulling="2026-05-21 11:50:22.073016872 +0000 UTC m=+1176.968300399" observedRunningTime="2026-05-21 11:50:22.803274422 +0000 UTC m=+1177.698557980" watchObservedRunningTime="2026-05-21 11:50:22.804278656 +0000 UTC m=+1177.699562216" May 21 11:50:27.012273 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:27.012238 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-7f849f6b56-fddfb"] May 21 11:50:27.021618 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:27.021596 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-fddfb" May 21 11:50:27.025550 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:27.025522 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" May 21 11:50:27.026477 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:27.026455 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-7f849f6b56-fddfb"] May 21 11:50:27.144507 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:27.144464 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/056966b8-b063-415a-841c-1e0719bf8266-model-cache\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-fddfb\" (UID: \"056966b8-b063-415a-841c-1e0719bf8266\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-fddfb" May 21 11:50:27.144507 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:27.144506 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/056966b8-b063-415a-841c-1e0719bf8266-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-fddfb\" (UID: \"056966b8-b063-415a-841c-1e0719bf8266\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-fddfb" May 21 11:50:27.144730 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:27.144538 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/056966b8-b063-415a-841c-1e0719bf8266-home\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-fddfb\" (UID: \"056966b8-b063-415a-841c-1e0719bf8266\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-fddfb" May 21 11:50:27.144730 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:27.144623 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjdpx\" (UniqueName: \"kubernetes.io/projected/056966b8-b063-415a-841c-1e0719bf8266-kube-api-access-pjdpx\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-fddfb\" (UID: \"056966b8-b063-415a-841c-1e0719bf8266\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-fddfb" May 21 11:50:27.144730 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:27.144662 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/056966b8-b063-415a-841c-1e0719bf8266-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-fddfb\" (UID: \"056966b8-b063-415a-841c-1e0719bf8266\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-fddfb" May 21 11:50:27.144730 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:27.144684 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/056966b8-b063-415a-841c-1e0719bf8266-dshm\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-fddfb\" (UID: \"056966b8-b063-415a-841c-1e0719bf8266\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-fddfb" May 21 11:50:27.245778 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:27.245738 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/056966b8-b063-415a-841c-1e0719bf8266-model-cache\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-fddfb\" (UID: \"056966b8-b063-415a-841c-1e0719bf8266\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-fddfb" May 21 11:50:27.245778 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:27.245778 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/056966b8-b063-415a-841c-1e0719bf8266-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-fddfb\" (UID: \"056966b8-b063-415a-841c-1e0719bf8266\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-fddfb" May 21 11:50:27.245999 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:27.245802 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/056966b8-b063-415a-841c-1e0719bf8266-home\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-fddfb\" (UID: \"056966b8-b063-415a-841c-1e0719bf8266\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-fddfb" May 21 11:50:27.245999 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:27.245840 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pjdpx\" (UniqueName: \"kubernetes.io/projected/056966b8-b063-415a-841c-1e0719bf8266-kube-api-access-pjdpx\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-fddfb\" (UID: \"056966b8-b063-415a-841c-1e0719bf8266\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-fddfb" May 21 11:50:27.245999 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:27.245869 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/056966b8-b063-415a-841c-1e0719bf8266-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-fddfb\" (UID: \"056966b8-b063-415a-841c-1e0719bf8266\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-fddfb" May 21 11:50:27.245999 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:27.245889 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/056966b8-b063-415a-841c-1e0719bf8266-dshm\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-fddfb\" (UID: \"056966b8-b063-415a-841c-1e0719bf8266\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-fddfb" May 21 11:50:27.246680 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:27.246652 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/056966b8-b063-415a-841c-1e0719bf8266-model-cache\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-fddfb\" (UID: \"056966b8-b063-415a-841c-1e0719bf8266\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-fddfb" May 21 11:50:27.246855 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:27.246673 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/056966b8-b063-415a-841c-1e0719bf8266-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-fddfb\" (UID: \"056966b8-b063-415a-841c-1e0719bf8266\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-fddfb" May 21 11:50:27.246990 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:27.246704 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/056966b8-b063-415a-841c-1e0719bf8266-home\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-fddfb\" (UID: \"056966b8-b063-415a-841c-1e0719bf8266\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-fddfb" May 21 11:50:27.249780 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:27.249758 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/056966b8-b063-415a-841c-1e0719bf8266-dshm\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-fddfb\" (UID: \"056966b8-b063-415a-841c-1e0719bf8266\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-fddfb" May 21 11:50:27.250152 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:27.250055 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/056966b8-b063-415a-841c-1e0719bf8266-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-fddfb\" (UID: \"056966b8-b063-415a-841c-1e0719bf8266\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-fddfb" May 21 11:50:27.256016 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:27.255996 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjdpx\" (UniqueName: \"kubernetes.io/projected/056966b8-b063-415a-841c-1e0719bf8266-kube-api-access-pjdpx\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-fddfb\" (UID: \"056966b8-b063-415a-841c-1e0719bf8266\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-fddfb" May 21 11:50:27.333465 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:27.333425 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-fddfb" May 21 11:50:27.486393 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:27.486368 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-7f849f6b56-fddfb"] May 21 11:50:27.487900 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:50:27.487866 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod056966b8_b063_415a_841c_1e0719bf8266.slice/crio-7b03514f091265054c409f0d31d6b58df1853fe19152da26ef9bed569d53349a WatchSource:0}: Error finding container 7b03514f091265054c409f0d31d6b58df1853fe19152da26ef9bed569d53349a: Status 404 returned error can't find the container with id 7b03514f091265054c409f0d31d6b58df1853fe19152da26ef9bed569d53349a May 21 11:50:27.793386 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:27.793292 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-fddfb" event={"ID":"056966b8-b063-415a-841c-1e0719bf8266","Type":"ContainerStarted","Data":"ba5eb66735991c4c2890e52aa5612046ab262a82a6c4ab5f08316d5cfe9b8dc8"} May 21 11:50:27.793386 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:27.793347 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-fddfb" event={"ID":"056966b8-b063-415a-841c-1e0719bf8266","Type":"ContainerStarted","Data":"7b03514f091265054c409f0d31d6b58df1853fe19152da26ef9bed569d53349a"} May 21 11:50:28.769282 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:28.769251 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-wrv26" May 21 11:50:32.814092 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:32.814040 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-6b97b89985-t84sl"] May 21 11:50:32.824187 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:32.824161 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-t84sl" May 21 11:50:32.828194 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:32.828167 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" May 21 11:50:32.828736 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:32.828716 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-6b97b89985-t84sl"] May 21 11:50:32.897053 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:32.897019 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/23fdb013-284c-499d-84dd-e3250531f1e4-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-t84sl\" (UID: \"23fdb013-284c-499d-84dd-e3250531f1e4\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-t84sl" May 21 11:50:32.897053 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:32.897057 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25b6b\" (UniqueName: \"kubernetes.io/projected/23fdb013-284c-499d-84dd-e3250531f1e4-kube-api-access-25b6b\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-t84sl\" (UID: \"23fdb013-284c-499d-84dd-e3250531f1e4\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-t84sl" May 21 11:50:32.905004 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:32.897581 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/23fdb013-284c-499d-84dd-e3250531f1e4-dshm\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-t84sl\" (UID: \"23fdb013-284c-499d-84dd-e3250531f1e4\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-t84sl" May 21 11:50:32.905281 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:32.905248 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23fdb013-284c-499d-84dd-e3250531f1e4-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-t84sl\" (UID: \"23fdb013-284c-499d-84dd-e3250531f1e4\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-t84sl" May 21 11:50:32.905564 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:32.905538 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/23fdb013-284c-499d-84dd-e3250531f1e4-home\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-t84sl\" (UID: \"23fdb013-284c-499d-84dd-e3250531f1e4\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-t84sl" May 21 11:50:32.905796 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:32.905755 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/23fdb013-284c-499d-84dd-e3250531f1e4-model-cache\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-t84sl\" (UID: \"23fdb013-284c-499d-84dd-e3250531f1e4\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-t84sl" May 21 11:50:33.007416 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:33.007373 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23fdb013-284c-499d-84dd-e3250531f1e4-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-t84sl\" (UID: \"23fdb013-284c-499d-84dd-e3250531f1e4\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-t84sl" May 21 11:50:33.007592 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:33.007429 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/23fdb013-284c-499d-84dd-e3250531f1e4-home\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-t84sl\" (UID: \"23fdb013-284c-499d-84dd-e3250531f1e4\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-t84sl" May 21 11:50:33.007592 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:33.007455 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/23fdb013-284c-499d-84dd-e3250531f1e4-model-cache\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-t84sl\" (UID: \"23fdb013-284c-499d-84dd-e3250531f1e4\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-t84sl" May 21 11:50:33.007592 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:33.007517 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/23fdb013-284c-499d-84dd-e3250531f1e4-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-t84sl\" (UID: \"23fdb013-284c-499d-84dd-e3250531f1e4\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-t84sl" May 21 11:50:33.007592 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:33.007544 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-25b6b\" (UniqueName: \"kubernetes.io/projected/23fdb013-284c-499d-84dd-e3250531f1e4-kube-api-access-25b6b\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-t84sl\" (UID: \"23fdb013-284c-499d-84dd-e3250531f1e4\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-t84sl" May 21 11:50:33.007794 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:33.007643 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/23fdb013-284c-499d-84dd-e3250531f1e4-dshm\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-t84sl\" (UID: \"23fdb013-284c-499d-84dd-e3250531f1e4\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-t84sl" May 21 11:50:33.007979 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:33.007950 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/23fdb013-284c-499d-84dd-e3250531f1e4-home\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-t84sl\" (UID: \"23fdb013-284c-499d-84dd-e3250531f1e4\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-t84sl" May 21 11:50:33.008066 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:33.007961 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/23fdb013-284c-499d-84dd-e3250531f1e4-model-cache\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-t84sl\" (UID: \"23fdb013-284c-499d-84dd-e3250531f1e4\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-t84sl" May 21 11:50:33.008336 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:33.008212 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/23fdb013-284c-499d-84dd-e3250531f1e4-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-t84sl\" (UID: \"23fdb013-284c-499d-84dd-e3250531f1e4\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-t84sl" May 21 11:50:33.010043 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:33.010023 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23fdb013-284c-499d-84dd-e3250531f1e4-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-t84sl\" (UID: \"23fdb013-284c-499d-84dd-e3250531f1e4\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-t84sl" May 21 11:50:33.010171 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:33.010124 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/23fdb013-284c-499d-84dd-e3250531f1e4-dshm\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-t84sl\" (UID: \"23fdb013-284c-499d-84dd-e3250531f1e4\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-t84sl" May 21 11:50:33.020485 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:33.020460 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-25b6b\" (UniqueName: \"kubernetes.io/projected/23fdb013-284c-499d-84dd-e3250531f1e4-kube-api-access-25b6b\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-t84sl\" (UID: \"23fdb013-284c-499d-84dd-e3250531f1e4\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-t84sl" May 21 11:50:33.136349 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:33.136319 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-t84sl" May 21 11:50:33.281379 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:33.281353 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-6b97b89985-t84sl"] May 21 11:50:33.283580 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:50:33.283551 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23fdb013_284c_499d_84dd_e3250531f1e4.slice/crio-1bcb6111dc7c15fec8da2647286ae1b56eb754026f138ee550d25da5e088565a WatchSource:0}: Error finding container 1bcb6111dc7c15fec8da2647286ae1b56eb754026f138ee550d25da5e088565a: Status 404 returned error can't find the container with id 1bcb6111dc7c15fec8da2647286ae1b56eb754026f138ee550d25da5e088565a May 21 11:50:33.798673 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:33.798643 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmfwzw" May 21 11:50:33.819306 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:33.819279 2564 generic.go:358] "Generic (PLEG): container finished" podID="056966b8-b063-415a-841c-1e0719bf8266" containerID="ba5eb66735991c4c2890e52aa5612046ab262a82a6c4ab5f08316d5cfe9b8dc8" exitCode=0 May 21 11:50:33.819694 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:33.819349 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-fddfb" event={"ID":"056966b8-b063-415a-841c-1e0719bf8266","Type":"ContainerDied","Data":"ba5eb66735991c4c2890e52aa5612046ab262a82a6c4ab5f08316d5cfe9b8dc8"} May 21 11:50:33.821582 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:33.821553 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-t84sl" event={"ID":"23fdb013-284c-499d-84dd-e3250531f1e4","Type":"ContainerStarted","Data":"b9778d91483c966e069211be4bc06a514de295933ffddf2406201c4dd9adef4a"} May 21 11:50:33.821707 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:33.821591 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-t84sl" event={"ID":"23fdb013-284c-499d-84dd-e3250531f1e4","Type":"ContainerStarted","Data":"1bcb6111dc7c15fec8da2647286ae1b56eb754026f138ee550d25da5e088565a"} May 21 11:50:34.828318 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:34.828271 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-fddfb" event={"ID":"056966b8-b063-415a-841c-1e0719bf8266","Type":"ContainerStarted","Data":"951d62a3a056247e06e3b548f93368b147ef4f8a79ec74d124534ae8857e3eab"} May 21 11:50:34.828717 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:34.828602 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-fddfb" May 21 11:50:34.852998 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:34.852951 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-fddfb" podStartSLOduration=8.572934183 podStartE2EDuration="8.852936353s" podCreationTimestamp="2026-05-21 11:50:26 +0000 UTC" firstStartedPulling="2026-05-21 11:50:33.819996836 +0000 UTC m=+1188.715280360" lastFinishedPulling="2026-05-21 11:50:34.099999005 +0000 UTC m=+1188.995282530" observedRunningTime="2026-05-21 11:50:34.850266359 +0000 UTC m=+1189.745549903" watchObservedRunningTime="2026-05-21 11:50:34.852936353 +0000 UTC m=+1189.748219898" May 21 11:50:41.795851 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:41.795816 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-nckp2"] May 21 11:50:41.802332 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:41.802309 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-nckp2" May 21 11:50:41.806263 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:41.806229 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" May 21 11:50:41.812708 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:41.812681 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-nckp2"] May 21 11:50:41.854662 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:41.854637 2564 generic.go:358] "Generic (PLEG): container finished" podID="23fdb013-284c-499d-84dd-e3250531f1e4" containerID="b9778d91483c966e069211be4bc06a514de295933ffddf2406201c4dd9adef4a" exitCode=0 May 21 11:50:41.854761 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:41.854673 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-t84sl" event={"ID":"23fdb013-284c-499d-84dd-e3250531f1e4","Type":"ContainerDied","Data":"b9778d91483c966e069211be4bc06a514de295933ffddf2406201c4dd9adef4a"} May 21 11:50:41.894225 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:41.894194 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f0e24af9-8811-4d2c-8df7-94ba4e848b0f-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-nckp2\" (UID: \"f0e24af9-8811-4d2c-8df7-94ba4e848b0f\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-nckp2" May 21 11:50:41.894394 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:41.894247 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0e24af9-8811-4d2c-8df7-94ba4e848b0f-model-cache\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-nckp2\" (UID: \"f0e24af9-8811-4d2c-8df7-94ba4e848b0f\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-nckp2" May 21 11:50:41.894394 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:41.894273 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0e24af9-8811-4d2c-8df7-94ba4e848b0f-tls-certs\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-nckp2\" (UID: \"f0e24af9-8811-4d2c-8df7-94ba4e848b0f\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-nckp2" May 21 11:50:41.894394 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:41.894366 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0e24af9-8811-4d2c-8df7-94ba4e848b0f-dshm\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-nckp2\" (UID: \"f0e24af9-8811-4d2c-8df7-94ba4e848b0f\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-nckp2" May 21 11:50:41.894592 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:41.894573 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0e24af9-8811-4d2c-8df7-94ba4e848b0f-home\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-nckp2\" (UID: \"f0e24af9-8811-4d2c-8df7-94ba4e848b0f\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-nckp2" May 21 11:50:41.894655 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:41.894616 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbfgt\" (UniqueName: \"kubernetes.io/projected/f0e24af9-8811-4d2c-8df7-94ba4e848b0f-kube-api-access-lbfgt\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-nckp2\" (UID: \"f0e24af9-8811-4d2c-8df7-94ba4e848b0f\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-nckp2" May 21 11:50:41.995853 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:41.995816 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f0e24af9-8811-4d2c-8df7-94ba4e848b0f-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-nckp2\" (UID: \"f0e24af9-8811-4d2c-8df7-94ba4e848b0f\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-nckp2" May 21 11:50:41.995853 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:41.995864 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0e24af9-8811-4d2c-8df7-94ba4e848b0f-model-cache\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-nckp2\" (UID: \"f0e24af9-8811-4d2c-8df7-94ba4e848b0f\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-nckp2" May 21 11:50:41.996119 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:41.995889 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0e24af9-8811-4d2c-8df7-94ba4e848b0f-tls-certs\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-nckp2\" (UID: \"f0e24af9-8811-4d2c-8df7-94ba4e848b0f\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-nckp2" May 21 11:50:41.996119 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:41.995928 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0e24af9-8811-4d2c-8df7-94ba4e848b0f-dshm\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-nckp2\" (UID: \"f0e24af9-8811-4d2c-8df7-94ba4e848b0f\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-nckp2" May 21 11:50:41.996119 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:41.996044 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0e24af9-8811-4d2c-8df7-94ba4e848b0f-home\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-nckp2\" (UID: \"f0e24af9-8811-4d2c-8df7-94ba4e848b0f\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-nckp2" May 21 11:50:41.996309 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:41.996279 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f0e24af9-8811-4d2c-8df7-94ba4e848b0f-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-nckp2\" (UID: \"f0e24af9-8811-4d2c-8df7-94ba4e848b0f\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-nckp2" May 21 11:50:41.996374 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:41.996308 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0e24af9-8811-4d2c-8df7-94ba4e848b0f-model-cache\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-nckp2\" (UID: \"f0e24af9-8811-4d2c-8df7-94ba4e848b0f\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-nckp2" May 21 11:50:41.996474 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:41.996444 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lbfgt\" (UniqueName: \"kubernetes.io/projected/f0e24af9-8811-4d2c-8df7-94ba4e848b0f-kube-api-access-lbfgt\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-nckp2\" (UID: \"f0e24af9-8811-4d2c-8df7-94ba4e848b0f\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-nckp2" May 21 11:50:41.996606 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:41.996486 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0e24af9-8811-4d2c-8df7-94ba4e848b0f-home\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-nckp2\" (UID: \"f0e24af9-8811-4d2c-8df7-94ba4e848b0f\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-nckp2" May 21 11:50:41.998358 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:41.998340 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0e24af9-8811-4d2c-8df7-94ba4e848b0f-dshm\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-nckp2\" (UID: \"f0e24af9-8811-4d2c-8df7-94ba4e848b0f\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-nckp2" May 21 11:50:41.998500 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:41.998483 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0e24af9-8811-4d2c-8df7-94ba4e848b0f-tls-certs\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-nckp2\" (UID: \"f0e24af9-8811-4d2c-8df7-94ba4e848b0f\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-nckp2" May 21 11:50:42.006035 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:42.006006 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbfgt\" (UniqueName: \"kubernetes.io/projected/f0e24af9-8811-4d2c-8df7-94ba4e848b0f-kube-api-access-lbfgt\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-nckp2\" (UID: \"f0e24af9-8811-4d2c-8df7-94ba4e848b0f\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-nckp2" May 21 11:50:42.116443 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:42.116411 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-nckp2" May 21 11:50:42.259192 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:42.259166 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-nckp2"] May 21 11:50:42.261572 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:50:42.261547 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0e24af9_8811_4d2c_8df7_94ba4e848b0f.slice/crio-952c6d08d7c7d572bcd59bea5fe5697ce60ffcb7e56b5424b46af0e7942c17c5 WatchSource:0}: Error finding container 952c6d08d7c7d572bcd59bea5fe5697ce60ffcb7e56b5424b46af0e7942c17c5: Status 404 returned error can't find the container with id 952c6d08d7c7d572bcd59bea5fe5697ce60ffcb7e56b5424b46af0e7942c17c5 May 21 11:50:42.860147 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:42.860107 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-t84sl" event={"ID":"23fdb013-284c-499d-84dd-e3250531f1e4","Type":"ContainerStarted","Data":"0c419bffba3d9571a32c68797b96af648a8fa082e204b4625ab6d08f2af8867d"} May 21 11:50:42.860600 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:42.860340 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-t84sl" May 21 11:50:42.861514 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:42.861491 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-nckp2" event={"ID":"f0e24af9-8811-4d2c-8df7-94ba4e848b0f","Type":"ContainerStarted","Data":"624c7e19a8df14a9b2bd0d1d72f1efa141d5847a954186003683c5bfb435c4b5"} May 21 11:50:42.861598 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:42.861521 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-nckp2" event={"ID":"f0e24af9-8811-4d2c-8df7-94ba4e848b0f","Type":"ContainerStarted","Data":"952c6d08d7c7d572bcd59bea5fe5697ce60ffcb7e56b5424b46af0e7942c17c5"} May 21 11:50:42.888989 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:42.888930 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-t84sl" podStartSLOduration=10.547851844 podStartE2EDuration="10.888911366s" podCreationTimestamp="2026-05-21 11:50:32 +0000 UTC" firstStartedPulling="2026-05-21 11:50:41.855265997 +0000 UTC m=+1196.750549521" lastFinishedPulling="2026-05-21 11:50:42.196325516 +0000 UTC m=+1197.091609043" observedRunningTime="2026-05-21 11:50:42.884764143 +0000 UTC m=+1197.780047702" watchObservedRunningTime="2026-05-21 11:50:42.888911366 +0000 UTC m=+1197.784194911" May 21 11:50:45.651385 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:45.651349 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-6g9xk_be463222-9e99-47cb-bf6b-525913ca9e29/console-operator/2.log" May 21 11:50:45.654704 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:45.654682 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-6g9xk_be463222-9e99-47cb-bf6b-525913ca9e29/console-operator/2.log" May 21 11:50:45.654835 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:45.654720 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sr2wt_45027dd2-2a4d-4ff5-b33f-48e3ee052186/ovn-acl-logging/0.log" May 21 11:50:45.658019 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:45.658002 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sr2wt_45027dd2-2a4d-4ff5-b33f-48e3ee052186/ovn-acl-logging/0.log" May 21 11:50:45.846426 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:45.846396 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-fddfb" May 21 11:50:46.695808 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:46.695767 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84db68679b-zzs8c"] May 21 11:50:46.702585 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:46.702559 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-zzs8c" May 21 11:50:46.705510 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:46.705488 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" May 21 11:50:46.709269 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:46.709244 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84db68679b-zzs8c"] May 21 11:50:46.742414 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:46.742383 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81080617-f6d8-48d6-97c9-2dbe9988480e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-zzs8c\" (UID: \"81080617-f6d8-48d6-97c9-2dbe9988480e\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-zzs8c" May 21 11:50:46.742599 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:46.742430 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqhcd\" (UniqueName: \"kubernetes.io/projected/81080617-f6d8-48d6-97c9-2dbe9988480e-kube-api-access-vqhcd\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-zzs8c\" (UID: \"81080617-f6d8-48d6-97c9-2dbe9988480e\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-zzs8c" May 21 11:50:46.742599 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:46.742501 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/81080617-f6d8-48d6-97c9-2dbe9988480e-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-zzs8c\" (UID: \"81080617-f6d8-48d6-97c9-2dbe9988480e\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-zzs8c" May 21 11:50:46.742599 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:46.742561 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81080617-f6d8-48d6-97c9-2dbe9988480e-home\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-zzs8c\" (UID: \"81080617-f6d8-48d6-97c9-2dbe9988480e\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-zzs8c" May 21 11:50:46.742732 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:46.742616 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81080617-f6d8-48d6-97c9-2dbe9988480e-dshm\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-zzs8c\" (UID: \"81080617-f6d8-48d6-97c9-2dbe9988480e\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-zzs8c" May 21 11:50:46.742732 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:46.742641 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81080617-f6d8-48d6-97c9-2dbe9988480e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-zzs8c\" (UID: \"81080617-f6d8-48d6-97c9-2dbe9988480e\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-zzs8c" May 21 11:50:46.843400 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:46.843362 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81080617-f6d8-48d6-97c9-2dbe9988480e-home\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-zzs8c\" (UID: \"81080617-f6d8-48d6-97c9-2dbe9988480e\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-zzs8c" May 21 11:50:46.843557 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:46.843427 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81080617-f6d8-48d6-97c9-2dbe9988480e-dshm\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-zzs8c\" (UID: \"81080617-f6d8-48d6-97c9-2dbe9988480e\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-zzs8c" May 21 11:50:46.843557 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:46.843450 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81080617-f6d8-48d6-97c9-2dbe9988480e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-zzs8c\" (UID: \"81080617-f6d8-48d6-97c9-2dbe9988480e\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-zzs8c" May 21 11:50:46.843557 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:46.843524 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81080617-f6d8-48d6-97c9-2dbe9988480e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-zzs8c\" (UID: \"81080617-f6d8-48d6-97c9-2dbe9988480e\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-zzs8c" May 21 11:50:46.843557 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:46.843546 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vqhcd\" (UniqueName: \"kubernetes.io/projected/81080617-f6d8-48d6-97c9-2dbe9988480e-kube-api-access-vqhcd\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-zzs8c\" (UID: \"81080617-f6d8-48d6-97c9-2dbe9988480e\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-zzs8c" May 21 11:50:46.843706 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:46.843576 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/81080617-f6d8-48d6-97c9-2dbe9988480e-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-zzs8c\" (UID: \"81080617-f6d8-48d6-97c9-2dbe9988480e\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-zzs8c" May 21 11:50:46.843748 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:46.843720 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81080617-f6d8-48d6-97c9-2dbe9988480e-home\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-zzs8c\" (UID: \"81080617-f6d8-48d6-97c9-2dbe9988480e\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-zzs8c" May 21 11:50:46.843894 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:46.843862 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/81080617-f6d8-48d6-97c9-2dbe9988480e-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-zzs8c\" (UID: \"81080617-f6d8-48d6-97c9-2dbe9988480e\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-zzs8c" May 21 11:50:46.844010 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:46.843871 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81080617-f6d8-48d6-97c9-2dbe9988480e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-zzs8c\" (UID: \"81080617-f6d8-48d6-97c9-2dbe9988480e\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-zzs8c" May 21 11:50:46.845689 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:46.845669 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81080617-f6d8-48d6-97c9-2dbe9988480e-dshm\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-zzs8c\" (UID: \"81080617-f6d8-48d6-97c9-2dbe9988480e\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-zzs8c" May 21 11:50:46.846091 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:46.846060 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81080617-f6d8-48d6-97c9-2dbe9988480e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-zzs8c\" (UID: \"81080617-f6d8-48d6-97c9-2dbe9988480e\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-zzs8c" May 21 11:50:46.852204 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:46.852185 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqhcd\" (UniqueName: \"kubernetes.io/projected/81080617-f6d8-48d6-97c9-2dbe9988480e-kube-api-access-vqhcd\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-zzs8c\" (UID: \"81080617-f6d8-48d6-97c9-2dbe9988480e\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-zzs8c" May 21 11:50:47.014455 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:47.014372 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-zzs8c" May 21 11:50:47.159193 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:47.159166 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84db68679b-zzs8c"] May 21 11:50:47.161003 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:50:47.160970 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81080617_f6d8_48d6_97c9_2dbe9988480e.slice/crio-c42da8d1a18c5abe68cba7adb1f6158e9bb9ba885846aad8f86b9ca83f654375 WatchSource:0}: Error finding container c42da8d1a18c5abe68cba7adb1f6158e9bb9ba885846aad8f86b9ca83f654375: Status 404 returned error can't find the container with id c42da8d1a18c5abe68cba7adb1f6158e9bb9ba885846aad8f86b9ca83f654375 May 21 11:50:47.882298 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:47.882256 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-zzs8c" event={"ID":"81080617-f6d8-48d6-97c9-2dbe9988480e","Type":"ContainerStarted","Data":"c1063ed2f0b903d42514ac089841ac0fc891667f1df094edcb74683f278f617a"} May 21 11:50:47.882741 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:47.882308 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-zzs8c" event={"ID":"81080617-f6d8-48d6-97c9-2dbe9988480e","Type":"ContainerStarted","Data":"c42da8d1a18c5abe68cba7adb1f6158e9bb9ba885846aad8f86b9ca83f654375"} May 21 11:50:47.883631 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:47.883601 2564 generic.go:358] "Generic (PLEG): container finished" podID="f0e24af9-8811-4d2c-8df7-94ba4e848b0f" containerID="624c7e19a8df14a9b2bd0d1d72f1efa141d5847a954186003683c5bfb435c4b5" exitCode=0 May 21 11:50:47.883725 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:47.883645 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-nckp2" event={"ID":"f0e24af9-8811-4d2c-8df7-94ba4e848b0f","Type":"ContainerDied","Data":"624c7e19a8df14a9b2bd0d1d72f1efa141d5847a954186003683c5bfb435c4b5"} May 21 11:50:48.889233 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:48.889196 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-nckp2" event={"ID":"f0e24af9-8811-4d2c-8df7-94ba4e848b0f","Type":"ContainerStarted","Data":"ea7c022851e6d34b93be865edc2aa7370445f5e8e0f89cb69c6ec411a44322d4"} May 21 11:50:48.920248 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:48.920191 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-nckp2" podStartSLOduration=7.649208886 podStartE2EDuration="7.920172657s" podCreationTimestamp="2026-05-21 11:50:41 +0000 UTC" firstStartedPulling="2026-05-21 11:50:47.884236902 +0000 UTC m=+1202.779520425" lastFinishedPulling="2026-05-21 11:50:48.155200661 +0000 UTC m=+1203.050484196" observedRunningTime="2026-05-21 11:50:48.919540624 +0000 UTC m=+1203.814824172" watchObservedRunningTime="2026-05-21 11:50:48.920172657 +0000 UTC m=+1203.815456201" May 21 11:50:52.906342 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:52.906309 2564 generic.go:358] "Generic (PLEG): container finished" podID="81080617-f6d8-48d6-97c9-2dbe9988480e" containerID="c1063ed2f0b903d42514ac089841ac0fc891667f1df094edcb74683f278f617a" exitCode=0 May 21 11:50:52.906701 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:52.906391 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-zzs8c" event={"ID":"81080617-f6d8-48d6-97c9-2dbe9988480e","Type":"ContainerDied","Data":"c1063ed2f0b903d42514ac089841ac0fc891667f1df094edcb74683f278f617a"} May 21 11:50:53.878260 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:53.878222 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-t84sl" May 21 11:50:53.912296 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:53.912259 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-zzs8c" event={"ID":"81080617-f6d8-48d6-97c9-2dbe9988480e","Type":"ContainerStarted","Data":"8cf309b79b1ca8860b36fd63c457b19886f275417e44191e4786b70b12821486"} May 21 11:50:53.912719 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:53.912506 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-zzs8c" May 21 11:50:53.936021 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:53.935977 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-zzs8c" podStartSLOduration=7.440025811 podStartE2EDuration="7.935964988s" podCreationTimestamp="2026-05-21 11:50:46 +0000 UTC" firstStartedPulling="2026-05-21 11:50:52.907274241 +0000 UTC m=+1207.802557780" lastFinishedPulling="2026-05-21 11:50:53.403213435 +0000 UTC m=+1208.298496957" observedRunningTime="2026-05-21 11:50:53.934068294 +0000 UTC m=+1208.829351839" watchObservedRunningTime="2026-05-21 11:50:53.935964988 +0000 UTC m=+1208.831248537" May 21 11:50:58.890224 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:58.890183 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-nckp2" May 21 11:50:58.902581 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:50:58.902555 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-nckp2" May 21 11:51:04.930765 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:51:04.930736 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-zzs8c" May 21 11:52:27.689875 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:52:27.689129 2564 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-79c6b74fcf-rs9b5"] May 21 11:52:27.689875 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:52:27.689487 2564 kuberuntime_container.go:864] "Killing container with a grace period" pod="opendatahub/maas-controller-79c6b74fcf-rs9b5" podUID="8b74d820-2c20-4991-b644-219c36189fec" containerName="manager" containerID="cri-o://257931ebee7857e766eb084dc649e96fae0f98df77289f7746ba3112cdcc8234" gracePeriod=30 May 21 11:52:27.938457 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:52:27.938434 2564 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79c6b74fcf-rs9b5" May 21 11:52:28.032014 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:52:28.031933 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c59j5\" (UniqueName: \"kubernetes.io/projected/8b74d820-2c20-4991-b644-219c36189fec-kube-api-access-c59j5\") pod \"8b74d820-2c20-4991-b644-219c36189fec\" (UID: \"8b74d820-2c20-4991-b644-219c36189fec\") " May 21 11:52:28.034167 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:52:28.034142 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b74d820-2c20-4991-b644-219c36189fec-kube-api-access-c59j5" (OuterVolumeSpecName: "kube-api-access-c59j5") pod "8b74d820-2c20-4991-b644-219c36189fec" (UID: "8b74d820-2c20-4991-b644-219c36189fec"). InnerVolumeSpecName "kube-api-access-c59j5". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 21 11:52:28.133291 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:52:28.133262 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c59j5\" (UniqueName: \"kubernetes.io/projected/8b74d820-2c20-4991-b644-219c36189fec-kube-api-access-c59j5\") on node \"ip-10-0-139-213.ec2.internal\" DevicePath \"\"" May 21 11:52:28.255617 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:52:28.255583 2564 generic.go:358] "Generic (PLEG): container finished" podID="8b74d820-2c20-4991-b644-219c36189fec" containerID="257931ebee7857e766eb084dc649e96fae0f98df77289f7746ba3112cdcc8234" exitCode=0 May 21 11:52:28.255771 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:52:28.255628 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79c6b74fcf-rs9b5" event={"ID":"8b74d820-2c20-4991-b644-219c36189fec","Type":"ContainerDied","Data":"257931ebee7857e766eb084dc649e96fae0f98df77289f7746ba3112cdcc8234"} May 21 11:52:28.255771 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:52:28.255651 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79c6b74fcf-rs9b5" event={"ID":"8b74d820-2c20-4991-b644-219c36189fec","Type":"ContainerDied","Data":"b32db0dbff5d7559865a21ad156701f5767e9381381c07ced6ee7d3fad828060"} May 21 11:52:28.255771 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:52:28.255652 2564 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79c6b74fcf-rs9b5" May 21 11:52:28.255771 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:52:28.255667 2564 scope.go:117] "RemoveContainer" containerID="257931ebee7857e766eb084dc649e96fae0f98df77289f7746ba3112cdcc8234" May 21 11:52:28.264579 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:52:28.264552 2564 scope.go:117] "RemoveContainer" containerID="257931ebee7857e766eb084dc649e96fae0f98df77289f7746ba3112cdcc8234" May 21 11:52:28.264847 ip-10-0-139-213 kubenswrapper[2564]: E0521 11:52:28.264827 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"257931ebee7857e766eb084dc649e96fae0f98df77289f7746ba3112cdcc8234\": container with ID starting with 257931ebee7857e766eb084dc649e96fae0f98df77289f7746ba3112cdcc8234 not found: ID does not exist" containerID="257931ebee7857e766eb084dc649e96fae0f98df77289f7746ba3112cdcc8234" May 21 11:52:28.264898 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:52:28.264855 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"257931ebee7857e766eb084dc649e96fae0f98df77289f7746ba3112cdcc8234"} err="failed to get container status \"257931ebee7857e766eb084dc649e96fae0f98df77289f7746ba3112cdcc8234\": rpc error: code = NotFound desc = could not find container \"257931ebee7857e766eb084dc649e96fae0f98df77289f7746ba3112cdcc8234\": container with ID starting with 257931ebee7857e766eb084dc649e96fae0f98df77289f7746ba3112cdcc8234 not found: ID does not exist" May 21 11:52:28.284472 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:52:28.284392 2564 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-79c6b74fcf-rs9b5"] May 21 11:52:28.290374 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:52:28.290353 2564 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-79c6b74fcf-rs9b5"] May 21 11:52:29.684394 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:52:29.684356 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b74d820-2c20-4991-b644-219c36189fec" path="/var/lib/kubelet/pods/8b74d820-2c20-4991-b644-219c36189fec/volumes" May 21 11:55:39.184283 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:39.184244 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-8687598f7d-6z85m_9b1b3588-0b02-4a55-907a-64b9513bf1df/maas-api/0.log" May 21 11:55:39.796367 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:39.796328 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c4fb87789-ppglg_52ff6c3d-2aae-4a8f-ba40-78b0afe91aa8/manager/0.log" May 21 11:55:42.765056 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:42.765018 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-df5b64c7d-w6jp2_5a2c2383-4064-47e5-9c12-32c47b6ef502/kube-auth-proxy/0.log" May 21 11:55:43.017412 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:43.017331 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58b6c8fdc7-f2wp8_74a00797-92ed-4a1e-83e6-4ec7f69eaec1/payload-processing/0.log" May 21 11:55:43.128202 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:43.128172 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-b7f67965d-272k2_4540e5a0-576b-4722-acfc-a2cec978f9e3/router/0.log" May 21 11:55:43.469794 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:43.469761 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-7f849f6b56-fddfb_056966b8-b063-415a-841c-1e0719bf8266/storage-initializer/0.log" May 21 11:55:43.478462 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:43.478435 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-7f849f6b56-fddfb_056966b8-b063-415a-841c-1e0719bf8266/main/0.log" May 21 11:55:43.587932 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:43.587898 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7bb4cdb4d7-nckp2_f0e24af9-8811-4d2c-8df7-94ba4e848b0f/storage-initializer/0.log" May 21 11:55:43.598112 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:43.598068 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7bb4cdb4d7-nckp2_f0e24af9-8811-4d2c-8df7-94ba4e848b0f/main/0.log" May 21 11:55:43.707094 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:43.707047 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-84db68679b-zzs8c_81080617-f6d8-48d6-97c9-2dbe9988480e/storage-initializer/0.log" May 21 11:55:43.715516 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:43.715493 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-84db68679b-zzs8c_81080617-f6d8-48d6-97c9-2dbe9988480e/main/0.log" May 21 11:55:43.834903 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:43.834879 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmfwzw_7eddd6e5-1cf0-4694-bdc9-ab88d755537a/storage-initializer/0.log" May 21 11:55:43.843525 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:43.843502 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmfwzw_7eddd6e5-1cf0-4694-bdc9-ab88d755537a/main/0.log" May 21 11:55:43.957677 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:43.957644 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-8f8dc67b7-wrv26_450a0d73-c37b-490e-9333-c311f927b961/storage-initializer/0.log" May 21 11:55:43.966562 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:43.966525 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-8f8dc67b7-wrv26_450a0d73-c37b-490e-9333-c311f927b961/main/0.log" May 21 11:55:44.077456 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:44.077420 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-6b97b89985-t84sl_23fdb013-284c-499d-84dd-e3250531f1e4/storage-initializer/0.log" May 21 11:55:44.085749 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:44.085680 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-6b97b89985-t84sl_23fdb013-284c-499d-84dd-e3250531f1e4/main/0.log" May 21 11:55:45.683235 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:45.683199 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-6g9xk_be463222-9e99-47cb-bf6b-525913ca9e29/console-operator/2.log" May 21 11:55:45.685906 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:45.685882 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-6g9xk_be463222-9e99-47cb-bf6b-525913ca9e29/console-operator/2.log" May 21 11:55:45.686927 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:45.686907 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sr2wt_45027dd2-2a4d-4ff5-b33f-48e3ee052186/ovn-acl-logging/0.log" May 21 11:55:45.688746 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:45.688726 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sr2wt_45027dd2-2a4d-4ff5-b33f-48e3ee052186/ovn-acl-logging/0.log" May 21 11:55:51.112938 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:51.112908 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-nlrhz_6b372f05-30c2-4e48-b411-9e1941308c2a/global-pull-secret-syncer/0.log" May 21 11:55:51.264486 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:51.264453 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-w8vhb_449c0696-add7-49fb-9892-5c100cdc6d7a/konnectivity-agent/0.log" May 21 11:55:51.354734 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:51.354695 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-213.ec2.internal_3da4ba4cdfe6fc200fbc8989e0319648/haproxy/0.log" May 21 11:55:56.917146 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:56.917042 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3ed3c3ab-07ec-404a-9f28-41e03eabaeaa/alertmanager/0.log" May 21 11:55:56.958691 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:56.958660 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3ed3c3ab-07ec-404a-9f28-41e03eabaeaa/config-reloader/0.log" May 21 11:55:56.997236 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:56.997208 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3ed3c3ab-07ec-404a-9f28-41e03eabaeaa/kube-rbac-proxy-web/0.log" May 21 11:55:57.029153 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:57.029130 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3ed3c3ab-07ec-404a-9f28-41e03eabaeaa/kube-rbac-proxy/0.log" May 21 11:55:57.068497 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:57.068468 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3ed3c3ab-07ec-404a-9f28-41e03eabaeaa/kube-rbac-proxy-metric/0.log" May 21 11:55:57.108858 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:57.108827 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3ed3c3ab-07ec-404a-9f28-41e03eabaeaa/prom-label-proxy/0.log" May 21 11:55:57.139956 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:57.139931 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3ed3c3ab-07ec-404a-9f28-41e03eabaeaa/init-config-reloader/0.log" May 21 11:55:57.514600 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:57.514574 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lc6jm_d5981a1d-940e-4b8c-b48b-5db4ba823683/node-exporter/0.log" May 21 11:55:57.540227 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:57.540200 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lc6jm_d5981a1d-940e-4b8c-b48b-5db4ba823683/kube-rbac-proxy/0.log" May 21 11:55:57.573566 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:57.573542 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lc6jm_d5981a1d-940e-4b8c-b48b-5db4ba823683/init-textfile/0.log" May 21 11:55:57.719510 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:57.719477 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-9c959559f-5m46j_3f352f56-2ade-4408-98eb-0384957496ef/kube-rbac-proxy-main/0.log" May 21 11:55:57.746807 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:57.746778 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-9c959559f-5m46j_3f352f56-2ade-4408-98eb-0384957496ef/kube-rbac-proxy-self/0.log" May 21 11:55:57.771655 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:57.771628 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-9c959559f-5m46j_3f352f56-2ade-4408-98eb-0384957496ef/openshift-state-metrics/0.log" May 21 11:55:57.811101 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:57.811047 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_951519f5-fcca-4472-8f27-d025f50e6156/prometheus/0.log" May 21 11:55:57.837275 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:57.837247 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_951519f5-fcca-4472-8f27-d025f50e6156/config-reloader/0.log" May 21 11:55:57.865514 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:57.865487 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_951519f5-fcca-4472-8f27-d025f50e6156/thanos-sidecar/0.log" May 21 11:55:57.891163 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:57.891132 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_951519f5-fcca-4472-8f27-d025f50e6156/kube-rbac-proxy-web/0.log" May 21 11:55:57.918033 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:57.918006 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_951519f5-fcca-4472-8f27-d025f50e6156/kube-rbac-proxy/0.log" May 21 11:55:57.943114 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:57.943027 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_951519f5-fcca-4472-8f27-d025f50e6156/kube-rbac-proxy-thanos/0.log" May 21 11:55:57.972093 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:57.972055 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_951519f5-fcca-4472-8f27-d025f50e6156/init-config-reloader/0.log" May 21 11:55:58.009093 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:58.009052 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7779bbdd78-ltkmn_0295ffc6-c516-4ec8-961e-3e7afcac0d6f/prometheus-operator/0.log" May 21 11:55:58.031218 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:58.031190 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7779bbdd78-ltkmn_0295ffc6-c516-4ec8-961e-3e7afcac0d6f/kube-rbac-proxy/0.log" May 21 11:55:58.064413 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:58.064387 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-6c4bbc67b7-7r4gg_c856948d-3c99-4f33-966f-ab0a2f08e2cc/prometheus-operator-admission-webhook/0.log" May 21 11:55:58.216616 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:58.216536 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-67f6f86555-zx427_473c66ed-1f18-4de7-a9f3-34be157198ec/thanos-query/0.log" May 21 11:55:58.248060 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:58.248033 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-67f6f86555-zx427_473c66ed-1f18-4de7-a9f3-34be157198ec/kube-rbac-proxy-web/0.log" May 21 11:55:58.298341 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:58.298309 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-67f6f86555-zx427_473c66ed-1f18-4de7-a9f3-34be157198ec/kube-rbac-proxy/0.log" May 21 11:55:58.332450 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:58.332426 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-67f6f86555-zx427_473c66ed-1f18-4de7-a9f3-34be157198ec/prom-label-proxy/0.log" May 21 11:55:58.369635 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:58.369606 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-67f6f86555-zx427_473c66ed-1f18-4de7-a9f3-34be157198ec/kube-rbac-proxy-rules/0.log" May 21 11:55:58.404425 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:58.404396 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-67f6f86555-zx427_473c66ed-1f18-4de7-a9f3-34be157198ec/kube-rbac-proxy-metrics/0.log" May 21 11:55:59.976475 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:59.976442 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-c5k8w/perf-node-gather-daemonset-lcxdn"] May 21 11:55:59.976879 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:59.976855 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b74d820-2c20-4991-b644-219c36189fec" containerName="manager" May 21 11:55:59.976879 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:59.976868 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b74d820-2c20-4991-b644-219c36189fec" containerName="manager" May 21 11:55:59.977001 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:59.976991 2564 memory_manager.go:356] "RemoveStaleState removing state" podUID="8b74d820-2c20-4991-b644-219c36189fec" containerName="manager" May 21 11:55:59.980427 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:59.980407 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-c5k8w/perf-node-gather-daemonset-lcxdn" May 21 11:55:59.984215 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:59.984197 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-c5k8w\"/\"openshift-service-ca.crt\"" May 21 11:55:59.984317 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:59.984198 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-c5k8w\"/\"kube-root-ca.crt\"" May 21 11:55:59.985682 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:59.985661 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-c5k8w\"/\"default-dockercfg-x5tj9\"" May 21 11:55:59.991093 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:55:59.991055 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-c5k8w/perf-node-gather-daemonset-lcxdn"] May 21 11:56:00.099505 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:56:00.099475 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6p5m\" (UniqueName: \"kubernetes.io/projected/7da0a24a-5c71-4d15-b712-0cb05f4ac6e5-kube-api-access-l6p5m\") pod \"perf-node-gather-daemonset-lcxdn\" (UID: \"7da0a24a-5c71-4d15-b712-0cb05f4ac6e5\") " pod="openshift-must-gather-c5k8w/perf-node-gather-daemonset-lcxdn" May 21 11:56:00.099759 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:56:00.099521 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7da0a24a-5c71-4d15-b712-0cb05f4ac6e5-sys\") pod \"perf-node-gather-daemonset-lcxdn\" (UID: \"7da0a24a-5c71-4d15-b712-0cb05f4ac6e5\") " pod="openshift-must-gather-c5k8w/perf-node-gather-daemonset-lcxdn" May 21 11:56:00.099759 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:56:00.099568 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7da0a24a-5c71-4d15-b712-0cb05f4ac6e5-proc\") pod \"perf-node-gather-daemonset-lcxdn\" (UID: \"7da0a24a-5c71-4d15-b712-0cb05f4ac6e5\") " pod="openshift-must-gather-c5k8w/perf-node-gather-daemonset-lcxdn" May 21 11:56:00.099759 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:56:00.099664 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7da0a24a-5c71-4d15-b712-0cb05f4ac6e5-podres\") pod \"perf-node-gather-daemonset-lcxdn\" (UID: \"7da0a24a-5c71-4d15-b712-0cb05f4ac6e5\") " pod="openshift-must-gather-c5k8w/perf-node-gather-daemonset-lcxdn" May 21 11:56:00.099759 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:56:00.099717 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7da0a24a-5c71-4d15-b712-0cb05f4ac6e5-lib-modules\") pod \"perf-node-gather-daemonset-lcxdn\" (UID: \"7da0a24a-5c71-4d15-b712-0cb05f4ac6e5\") " pod="openshift-must-gather-c5k8w/perf-node-gather-daemonset-lcxdn" May 21 11:56:00.185786 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:56:00.185753 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-6g9xk_be463222-9e99-47cb-bf6b-525913ca9e29/console-operator/2.log" May 21 11:56:00.193139 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:56:00.193104 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-6g9xk_be463222-9e99-47cb-bf6b-525913ca9e29/console-operator/3.log" May 21 11:56:00.200345 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:56:00.200319 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7da0a24a-5c71-4d15-b712-0cb05f4ac6e5-podres\") pod \"perf-node-gather-daemonset-lcxdn\" (UID: \"7da0a24a-5c71-4d15-b712-0cb05f4ac6e5\") " pod="openshift-must-gather-c5k8w/perf-node-gather-daemonset-lcxdn" May 21 11:56:00.200458 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:56:00.200366 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7da0a24a-5c71-4d15-b712-0cb05f4ac6e5-lib-modules\") pod \"perf-node-gather-daemonset-lcxdn\" (UID: \"7da0a24a-5c71-4d15-b712-0cb05f4ac6e5\") " pod="openshift-must-gather-c5k8w/perf-node-gather-daemonset-lcxdn" May 21 11:56:00.200515 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:56:00.200453 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-l6p5m\" (UniqueName: \"kubernetes.io/projected/7da0a24a-5c71-4d15-b712-0cb05f4ac6e5-kube-api-access-l6p5m\") pod \"perf-node-gather-daemonset-lcxdn\" (UID: \"7da0a24a-5c71-4d15-b712-0cb05f4ac6e5\") " pod="openshift-must-gather-c5k8w/perf-node-gather-daemonset-lcxdn" May 21 11:56:00.200515 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:56:00.200484 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7da0a24a-5c71-4d15-b712-0cb05f4ac6e5-podres\") pod \"perf-node-gather-daemonset-lcxdn\" (UID: \"7da0a24a-5c71-4d15-b712-0cb05f4ac6e5\") " pod="openshift-must-gather-c5k8w/perf-node-gather-daemonset-lcxdn" May 21 11:56:00.200515 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:56:00.200496 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7da0a24a-5c71-4d15-b712-0cb05f4ac6e5-sys\") pod \"perf-node-gather-daemonset-lcxdn\" (UID: \"7da0a24a-5c71-4d15-b712-0cb05f4ac6e5\") " pod="openshift-must-gather-c5k8w/perf-node-gather-daemonset-lcxdn" May 21 11:56:00.200641 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:56:00.200547 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7da0a24a-5c71-4d15-b712-0cb05f4ac6e5-lib-modules\") pod \"perf-node-gather-daemonset-lcxdn\" (UID: \"7da0a24a-5c71-4d15-b712-0cb05f4ac6e5\") " pod="openshift-must-gather-c5k8w/perf-node-gather-daemonset-lcxdn" May 21 11:56:00.200641 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:56:00.200551 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7da0a24a-5c71-4d15-b712-0cb05f4ac6e5-sys\") pod \"perf-node-gather-daemonset-lcxdn\" (UID: \"7da0a24a-5c71-4d15-b712-0cb05f4ac6e5\") " pod="openshift-must-gather-c5k8w/perf-node-gather-daemonset-lcxdn" May 21 11:56:00.200641 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:56:00.200597 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7da0a24a-5c71-4d15-b712-0cb05f4ac6e5-proc\") pod \"perf-node-gather-daemonset-lcxdn\" (UID: \"7da0a24a-5c71-4d15-b712-0cb05f4ac6e5\") " pod="openshift-must-gather-c5k8w/perf-node-gather-daemonset-lcxdn" May 21 11:56:00.200745 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:56:00.200672 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7da0a24a-5c71-4d15-b712-0cb05f4ac6e5-proc\") pod \"perf-node-gather-daemonset-lcxdn\" (UID: \"7da0a24a-5c71-4d15-b712-0cb05f4ac6e5\") " pod="openshift-must-gather-c5k8w/perf-node-gather-daemonset-lcxdn" May 21 11:56:00.211604 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:56:00.211575 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6p5m\" (UniqueName: \"kubernetes.io/projected/7da0a24a-5c71-4d15-b712-0cb05f4ac6e5-kube-api-access-l6p5m\") pod \"perf-node-gather-daemonset-lcxdn\" (UID: \"7da0a24a-5c71-4d15-b712-0cb05f4ac6e5\") " pod="openshift-must-gather-c5k8w/perf-node-gather-daemonset-lcxdn" May 21 11:56:00.291187 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:56:00.291102 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-c5k8w/perf-node-gather-daemonset-lcxdn" May 21 11:56:00.428493 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:56:00.428454 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-c5k8w/perf-node-gather-daemonset-lcxdn"] May 21 11:56:00.430497 ip-10-0-139-213 kubenswrapper[2564]: W0521 11:56:00.430462 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod7da0a24a_5c71_4d15_b712_0cb05f4ac6e5.slice/crio-e931ca7ccc3ed22defe3880867b4403bb778bda45233e4344cdcc6c33ee2fed2 WatchSource:0}: Error finding container e931ca7ccc3ed22defe3880867b4403bb778bda45233e4344cdcc6c33ee2fed2: Status 404 returned error can't find the container with id e931ca7ccc3ed22defe3880867b4403bb778bda45233e4344cdcc6c33ee2fed2 May 21 11:56:00.432251 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:56:00.432232 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 21 11:56:01.029409 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:56:01.029369 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-c5k8w/perf-node-gather-daemonset-lcxdn" event={"ID":"7da0a24a-5c71-4d15-b712-0cb05f4ac6e5","Type":"ContainerStarted","Data":"a347a294ad15d20537b564e3dbd95d851066331dcb0736fe18dddf48e5d6025c"} May 21 11:56:01.029409 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:56:01.029410 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-c5k8w/perf-node-gather-daemonset-lcxdn" event={"ID":"7da0a24a-5c71-4d15-b712-0cb05f4ac6e5","Type":"ContainerStarted","Data":"e931ca7ccc3ed22defe3880867b4403bb778bda45233e4344cdcc6c33ee2fed2"} May 21 11:56:01.029865 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:56:01.029488 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-c5k8w/perf-node-gather-daemonset-lcxdn" May 21 11:56:01.048314 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:56:01.048269 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-c5k8w/perf-node-gather-daemonset-lcxdn" podStartSLOduration=2.04825739 podStartE2EDuration="2.04825739s" podCreationTimestamp="2026-05-21 11:55:59 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-21 11:56:01.047531418 +0000 UTC m=+1515.942814967" watchObservedRunningTime="2026-05-21 11:56:01.04825739 +0000 UTC m=+1515.943540935" May 21 11:56:02.147959 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:56:02.147933 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2pfk6_2190640c-a631-4c98-8d97-ade39bb1567a/dns/0.log" May 21 11:56:02.172330 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:56:02.172299 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2pfk6_2190640c-a631-4c98-8d97-ade39bb1567a/kube-rbac-proxy/0.log" May 21 11:56:02.342962 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:56:02.342934 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lkfnm_71722934-df74-4aa7-91e7-9f388d432991/dns-node-resolver/0.log" May 21 11:56:02.873625 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:56:02.873595 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-29k74_0fd57831-cff3-4f96-8695-cb4d02bc95a9/node-ca/0.log" May 21 11:56:04.016981 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:56:04.016952 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-df5b64c7d-w6jp2_5a2c2383-4064-47e5-9c12-32c47b6ef502/kube-auth-proxy/0.log" May 21 11:56:04.103625 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:56:04.103596 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58b6c8fdc7-f2wp8_74a00797-92ed-4a1e-83e6-4ec7f69eaec1/payload-processing/0.log" May 21 11:56:04.129934 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:56:04.129905 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-b7f67965d-272k2_4540e5a0-576b-4722-acfc-a2cec978f9e3/router/0.log" May 21 11:56:04.742535 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:56:04.742507 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-s79qs_b14cbac4-0b39-404a-a413-a5ccf5fb6472/serve-healthcheck-canary/0.log" May 21 11:56:05.207298 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:56:05.207270 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-78bdfb888c-49stn_f797c9ec-815c-4e1a-aa83-9173927d8593/insights-operator/0.log" May 21 11:56:05.207757 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:56:05.207568 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-78bdfb888c-49stn_f797c9ec-815c-4e1a-aa83-9173927d8593/insights-operator/1.log" May 21 11:56:05.312385 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:56:05.312355 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jq45s_684e8f1e-9fe0-43a7-8c45-6ae9fde9e165/kube-rbac-proxy/0.log" May 21 11:56:05.336518 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:56:05.336489 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jq45s_684e8f1e-9fe0-43a7-8c45-6ae9fde9e165/exporter/0.log" May 21 11:56:05.361826 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:56:05.361805 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jq45s_684e8f1e-9fe0-43a7-8c45-6ae9fde9e165/extractor/0.log" May 21 11:56:07.043701 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:56:07.043676 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-c5k8w/perf-node-gather-daemonset-lcxdn" May 21 11:56:07.580896 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:56:07.580866 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-8687598f7d-6z85m_9b1b3588-0b02-4a55-907a-64b9513bf1df/maas-api/0.log" May 21 11:56:07.777067 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:56:07.777032 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c4fb87789-ppglg_52ff6c3d-2aae-4a8f-ba40-78b0afe91aa8/manager/0.log" May 21 11:56:09.155282 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:56:09.155247 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5fd9cb47f5-pvxcn_1bfd0a23-d936-4c65-a519-77218cd5f6f3/manager/0.log" May 21 11:56:14.151230 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:56:14.151198 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-565644f49-xdnz5_a74a4dc1-ca52-40f4-9cb2-0c925a08ecfd/kube-storage-version-migrator-operator/1.log" May 21 11:56:14.152876 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:56:14.152842 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-565644f49-xdnz5_a74a4dc1-ca52-40f4-9cb2-0c925a08ecfd/kube-storage-version-migrator-operator/0.log" May 21 11:56:15.264067 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:56:15.264037 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9nh44_bfa04605-9b0c-4c9b-a808-598623c60d2d/kube-multus-additional-cni-plugins/0.log" May 21 11:56:15.292239 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:56:15.292215 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9nh44_bfa04605-9b0c-4c9b-a808-598623c60d2d/egress-router-binary-copy/0.log" May 21 11:56:15.316851 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:56:15.316816 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9nh44_bfa04605-9b0c-4c9b-a808-598623c60d2d/cni-plugins/0.log" May 21 11:56:15.341286 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:56:15.341253 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9nh44_bfa04605-9b0c-4c9b-a808-598623c60d2d/bond-cni-plugin/0.log" May 21 11:56:15.364695 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:56:15.364667 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9nh44_bfa04605-9b0c-4c9b-a808-598623c60d2d/routeoverride-cni/0.log" May 21 11:56:15.390089 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:56:15.390059 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9nh44_bfa04605-9b0c-4c9b-a808-598623c60d2d/whereabouts-cni-bincopy/0.log" May 21 11:56:15.413734 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:56:15.413713 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9nh44_bfa04605-9b0c-4c9b-a808-598623c60d2d/whereabouts-cni/0.log" May 21 11:56:15.819465 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:56:15.819434 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-q4k9w_fceead7c-3987-4250-816f-e89182d3b315/kube-multus/0.log" May 21 11:56:15.959920 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:56:15.959884 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-xf5gc_414e12fe-b9ac-4bbf-8470-ad442215f5cb/network-metrics-daemon/0.log" May 21 11:56:15.982897 ip-10-0-139-213 kubenswrapper[2564]: I0521 11:56:15.982854 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-xf5gc_414e12fe-b9ac-4bbf-8470-ad442215f5cb/kube-rbac-proxy/0.log"