May 27 21:26:06.224437 ip-10-0-130-246 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory May 27 21:26:06.224447 ip-10-0-130-246 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory May 27 21:26:06.224455 ip-10-0-130-246 systemd[1]: kubelet.service: Failed with result 'resources'. May 27 21:26:06.224658 ip-10-0-130-246 systemd[1]: Failed to start Kubernetes Kubelet. May 27 21:26:17.720076 ip-10-0-130-246 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. May 27 21:26:17.720091 ip-10-0-130-246 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot ee8b0377edce4e34b441a2535ca9136d -- May 27 21:28:46.409584 ip-10-0-130-246 systemd[1]: Starting Kubernetes Kubelet... May 27 21:28:46.834784 ip-10-0-130-246 kubenswrapper[2564]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. May 27 21:28:46.834784 ip-10-0-130-246 kubenswrapper[2564]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. May 27 21:28:46.834784 ip-10-0-130-246 kubenswrapper[2564]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. May 27 21:28:46.834784 ip-10-0-130-246 kubenswrapper[2564]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. May 27 21:28:46.834784 ip-10-0-130-246 kubenswrapper[2564]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. May 27 21:28:46.836669 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.836529 2564 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" May 27 21:28:46.840953 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.840938 2564 feature_gate.go:328] unrecognized feature gate: DualReplica May 27 21:28:46.840953 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.840953 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 27 21:28:46.841016 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.840957 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 27 21:28:46.841016 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.840960 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 27 21:28:46.841016 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.840965 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 27 21:28:46.841016 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.840968 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 27 21:28:46.841016 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.840971 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 27 21:28:46.841016 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.840974 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 27 21:28:46.841016 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.840983 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 27 21:28:46.841016 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.840987 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 27 21:28:46.841016 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.840989 2564 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 27 21:28:46.841016 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.840992 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 27 21:28:46.841016 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.840995 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 27 21:28:46.841016 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.840997 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 27 21:28:46.841016 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841000 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 27 21:28:46.841016 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841003 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 27 21:28:46.841016 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841005 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 27 21:28:46.841016 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841007 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 27 21:28:46.841016 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841010 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 27 21:28:46.841016 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841012 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 27 21:28:46.841016 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841015 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 27 21:28:46.841516 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841018 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 27 21:28:46.841516 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841020 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 27 21:28:46.841516 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841023 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 27 21:28:46.841516 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841026 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 27 21:28:46.841516 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841030 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 27 21:28:46.841516 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841033 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages May 27 21:28:46.841516 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841036 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 27 21:28:46.841516 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841039 2564 feature_gate.go:328] unrecognized feature gate: NewOLM May 27 21:28:46.841516 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841041 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 27 21:28:46.841516 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841043 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 27 21:28:46.841516 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841046 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 27 21:28:46.841516 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841048 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 27 21:28:46.841516 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841050 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 27 21:28:46.841516 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841053 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 27 21:28:46.841516 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841055 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 27 21:28:46.841516 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841060 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 27 21:28:46.841516 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841063 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 27 21:28:46.841516 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841066 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 27 21:28:46.841516 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841069 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 27 21:28:46.842035 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841071 2564 feature_gate.go:328] unrecognized feature gate: Example2 May 27 21:28:46.842035 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841075 2564 feature_gate.go:328] unrecognized feature gate: Example May 27 21:28:46.842035 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841077 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 27 21:28:46.842035 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841080 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 27 21:28:46.842035 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841083 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 27 21:28:46.842035 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841085 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 27 21:28:46.842035 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841087 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 27 21:28:46.842035 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841090 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 27 21:28:46.842035 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841092 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 27 21:28:46.842035 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841095 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability May 27 21:28:46.842035 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841097 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 27 21:28:46.842035 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841100 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 27 21:28:46.842035 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841102 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 27 21:28:46.842035 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841104 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 27 21:28:46.842035 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841109 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 27 21:28:46.842035 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841111 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 27 21:28:46.842035 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841114 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 27 21:28:46.842035 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841117 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 27 21:28:46.842035 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841120 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 27 21:28:46.842035 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841122 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 27 21:28:46.842531 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841124 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 27 21:28:46.842531 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841127 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 27 21:28:46.842531 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841129 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 27 21:28:46.842531 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841132 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 27 21:28:46.842531 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841134 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 27 21:28:46.842531 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841137 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 27 21:28:46.842531 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841139 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 27 21:28:46.842531 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841141 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 27 21:28:46.842531 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841144 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 27 21:28:46.842531 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841146 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 27 21:28:46.842531 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841148 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 27 21:28:46.842531 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841151 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores May 27 21:28:46.842531 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841153 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 27 21:28:46.842531 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841155 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 27 21:28:46.842531 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841158 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 27 21:28:46.842531 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841161 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 27 21:28:46.842531 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841163 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 27 21:28:46.842531 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841165 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 27 21:28:46.842531 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841168 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 27 21:28:46.842531 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841170 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 27 21:28:46.843010 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841172 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 27 21:28:46.843010 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841175 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 27 21:28:46.843010 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841177 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 27 21:28:46.843010 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841180 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 27 21:28:46.843010 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841182 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 27 21:28:46.843010 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841184 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 27 21:28:46.843010 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841554 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 27 21:28:46.843010 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841559 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 27 21:28:46.843010 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841562 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 27 21:28:46.843010 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841565 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 27 21:28:46.843010 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841567 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 27 21:28:46.843010 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841570 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 27 21:28:46.843010 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841573 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 27 21:28:46.843010 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841575 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 27 21:28:46.843010 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841578 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 27 21:28:46.843010 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841580 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 27 21:28:46.843010 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841583 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 27 21:28:46.843010 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841585 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 27 21:28:46.843010 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841588 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 27 21:28:46.843010 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841590 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 27 21:28:46.843513 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841592 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 27 21:28:46.843513 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841595 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 27 21:28:46.843513 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841597 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 27 21:28:46.843513 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841600 2564 feature_gate.go:328] unrecognized feature gate: NewOLM May 27 21:28:46.843513 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841602 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 27 21:28:46.843513 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841605 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 27 21:28:46.843513 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841610 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 27 21:28:46.843513 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841613 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 27 21:28:46.843513 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841616 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 27 21:28:46.843513 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841620 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 27 21:28:46.843513 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841623 2564 feature_gate.go:328] unrecognized feature gate: DualReplica May 27 21:28:46.843513 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841625 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 27 21:28:46.843513 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841628 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 27 21:28:46.843513 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841630 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 27 21:28:46.843513 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841633 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 27 21:28:46.843513 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841635 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 27 21:28:46.843513 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841637 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 27 21:28:46.843513 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841640 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 27 21:28:46.843513 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841644 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 27 21:28:46.844008 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841646 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 27 21:28:46.844008 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841649 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 27 21:28:46.844008 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841652 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 27 21:28:46.844008 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841655 2564 feature_gate.go:328] unrecognized feature gate: Example May 27 21:28:46.844008 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841657 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 27 21:28:46.844008 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841660 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 27 21:28:46.844008 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841663 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 27 21:28:46.844008 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841665 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 27 21:28:46.844008 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841668 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 27 21:28:46.844008 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841671 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 27 21:28:46.844008 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841673 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 27 21:28:46.844008 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841676 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 27 21:28:46.844008 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841678 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability May 27 21:28:46.844008 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841680 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 27 21:28:46.844008 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841683 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 27 21:28:46.844008 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841686 2564 feature_gate.go:328] unrecognized feature gate: Example2 May 27 21:28:46.844008 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841688 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 27 21:28:46.844008 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841691 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 27 21:28:46.844008 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841693 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 27 21:28:46.844008 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841696 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 27 21:28:46.844496 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841699 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 27 21:28:46.844496 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841701 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 27 21:28:46.844496 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841704 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 27 21:28:46.844496 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841707 2564 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 27 21:28:46.844496 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841710 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 27 21:28:46.844496 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841712 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 27 21:28:46.844496 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841716 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 27 21:28:46.844496 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841718 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 27 21:28:46.844496 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841721 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 27 21:28:46.844496 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841723 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 27 21:28:46.844496 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841726 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 27 21:28:46.844496 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841729 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores May 27 21:28:46.844496 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841732 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 27 21:28:46.844496 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841734 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 27 21:28:46.844496 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841737 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 27 21:28:46.844496 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841739 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 27 21:28:46.844496 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841742 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 27 21:28:46.844496 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841745 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 27 21:28:46.844496 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841747 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 27 21:28:46.844496 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841749 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 27 21:28:46.844968 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841752 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 27 21:28:46.844968 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841756 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 27 21:28:46.844968 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841758 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 27 21:28:46.844968 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841760 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 27 21:28:46.844968 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841763 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 27 21:28:46.844968 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841765 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 27 21:28:46.844968 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841768 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages May 27 21:28:46.844968 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841770 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 27 21:28:46.844968 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841773 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 27 21:28:46.844968 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841776 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 27 21:28:46.844968 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841778 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 27 21:28:46.844968 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841781 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 27 21:28:46.844968 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.841784 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 27 21:28:46.844968 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843091 2564 flags.go:64] FLAG: --address="0.0.0.0" May 27 21:28:46.844968 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843100 2564 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" May 27 21:28:46.844968 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843115 2564 flags.go:64] FLAG: --anonymous-auth="true" May 27 21:28:46.844968 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843124 2564 flags.go:64] FLAG: --application-metrics-count-limit="100" May 27 21:28:46.844968 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843128 2564 flags.go:64] FLAG: --authentication-token-webhook="false" May 27 21:28:46.844968 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843132 2564 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" May 27 21:28:46.844968 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843136 2564 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" May 27 21:28:46.844968 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843141 2564 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" May 27 21:28:46.845483 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843144 2564 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" May 27 21:28:46.845483 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843147 2564 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" May 27 21:28:46.845483 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843151 2564 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" May 27 21:28:46.845483 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843154 2564 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" May 27 21:28:46.845483 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843157 2564 flags.go:64] FLAG: --cgroup-driver="cgroupfs" May 27 21:28:46.845483 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843160 2564 flags.go:64] FLAG: --cgroup-root="" May 27 21:28:46.845483 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843163 2564 flags.go:64] FLAG: --cgroups-per-qos="true" May 27 21:28:46.845483 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843166 2564 flags.go:64] FLAG: --client-ca-file="" May 27 21:28:46.845483 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843169 2564 flags.go:64] FLAG: --cloud-config="" May 27 21:28:46.845483 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843172 2564 flags.go:64] FLAG: --cloud-provider="external" May 27 21:28:46.845483 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843175 2564 flags.go:64] FLAG: --cluster-dns="[]" May 27 21:28:46.845483 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843179 2564 flags.go:64] FLAG: --cluster-domain="" May 27 21:28:46.845483 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843182 2564 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" May 27 21:28:46.845483 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843185 2564 flags.go:64] FLAG: --config-dir="" May 27 21:28:46.845483 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843187 2564 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" May 27 21:28:46.845483 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843191 2564 flags.go:64] FLAG: --container-log-max-files="5" May 27 21:28:46.845483 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843195 2564 flags.go:64] FLAG: --container-log-max-size="10Mi" May 27 21:28:46.845483 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843198 2564 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" May 27 21:28:46.845483 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843201 2564 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" May 27 21:28:46.845483 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843204 2564 flags.go:64] FLAG: --containerd-namespace="k8s.io" May 27 21:28:46.845483 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843207 2564 flags.go:64] FLAG: --contention-profiling="false" May 27 21:28:46.845483 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843210 2564 flags.go:64] FLAG: --cpu-cfs-quota="true" May 27 21:28:46.845483 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843213 2564 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" May 27 21:28:46.845483 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843216 2564 flags.go:64] FLAG: --cpu-manager-policy="none" May 27 21:28:46.845483 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843218 2564 flags.go:64] FLAG: --cpu-manager-policy-options="" May 27 21:28:46.846082 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843223 2564 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" May 27 21:28:46.846082 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843226 2564 flags.go:64] FLAG: --enable-controller-attach-detach="true" May 27 21:28:46.846082 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843229 2564 flags.go:64] FLAG: --enable-debugging-handlers="true" May 27 21:28:46.846082 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843232 2564 flags.go:64] FLAG: --enable-load-reader="false" May 27 21:28:46.846082 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843235 2564 flags.go:64] FLAG: --enable-server="true" May 27 21:28:46.846082 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843238 2564 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" May 27 21:28:46.846082 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843243 2564 flags.go:64] FLAG: --event-burst="100" May 27 21:28:46.846082 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843246 2564 flags.go:64] FLAG: --event-qps="50" May 27 21:28:46.846082 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843249 2564 flags.go:64] FLAG: --event-storage-age-limit="default=0" May 27 21:28:46.846082 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843252 2564 flags.go:64] FLAG: --event-storage-event-limit="default=0" May 27 21:28:46.846082 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843255 2564 flags.go:64] FLAG: --eviction-hard="" May 27 21:28:46.846082 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843259 2564 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" May 27 21:28:46.846082 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843275 2564 flags.go:64] FLAG: --eviction-minimum-reclaim="" May 27 21:28:46.846082 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843278 2564 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" May 27 21:28:46.846082 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843281 2564 flags.go:64] FLAG: --eviction-soft="" May 27 21:28:46.846082 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843284 2564 flags.go:64] FLAG: --eviction-soft-grace-period="" May 27 21:28:46.846082 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843287 2564 flags.go:64] FLAG: --exit-on-lock-contention="false" May 27 21:28:46.846082 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843290 2564 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" May 27 21:28:46.846082 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843293 2564 flags.go:64] FLAG: --experimental-mounter-path="" May 27 21:28:46.846082 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843296 2564 flags.go:64] FLAG: --fail-cgroupv1="false" May 27 21:28:46.846082 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843299 2564 flags.go:64] FLAG: --fail-swap-on="true" May 27 21:28:46.846082 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843301 2564 flags.go:64] FLAG: --feature-gates="" May 27 21:28:46.846082 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843305 2564 flags.go:64] FLAG: --file-check-frequency="20s" May 27 21:28:46.846082 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843308 2564 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" May 27 21:28:46.846082 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843312 2564 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" May 27 21:28:46.846684 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843315 2564 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" May 27 21:28:46.846684 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843318 2564 flags.go:64] FLAG: --healthz-port="10248" May 27 21:28:46.846684 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843321 2564 flags.go:64] FLAG: --help="false" May 27 21:28:46.846684 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843325 2564 flags.go:64] FLAG: --hostname-override="ip-10-0-130-246.ec2.internal" May 27 21:28:46.846684 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843328 2564 flags.go:64] FLAG: --housekeeping-interval="10s" May 27 21:28:46.846684 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843331 2564 flags.go:64] FLAG: --http-check-frequency="20s" May 27 21:28:46.846684 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843334 2564 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" May 27 21:28:46.846684 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843337 2564 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" May 27 21:28:46.846684 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843341 2564 flags.go:64] FLAG: --image-gc-high-threshold="85" May 27 21:28:46.846684 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843344 2564 flags.go:64] FLAG: --image-gc-low-threshold="80" May 27 21:28:46.846684 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843347 2564 flags.go:64] FLAG: --image-service-endpoint="" May 27 21:28:46.846684 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843350 2564 flags.go:64] FLAG: --kernel-memcg-notification="false" May 27 21:28:46.846684 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843353 2564 flags.go:64] FLAG: --kube-api-burst="100" May 27 21:28:46.846684 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843356 2564 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" May 27 21:28:46.846684 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843359 2564 flags.go:64] FLAG: --kube-api-qps="50" May 27 21:28:46.846684 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843361 2564 flags.go:64] FLAG: --kube-reserved="" May 27 21:28:46.846684 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843364 2564 flags.go:64] FLAG: --kube-reserved-cgroup="" May 27 21:28:46.846684 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843367 2564 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" May 27 21:28:46.846684 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843370 2564 flags.go:64] FLAG: --kubelet-cgroups="" May 27 21:28:46.846684 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843373 2564 flags.go:64] FLAG: --local-storage-capacity-isolation="true" May 27 21:28:46.846684 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843376 2564 flags.go:64] FLAG: --lock-file="" May 27 21:28:46.846684 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843378 2564 flags.go:64] FLAG: --log-cadvisor-usage="false" May 27 21:28:46.846684 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843381 2564 flags.go:64] FLAG: --log-flush-frequency="5s" May 27 21:28:46.846684 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843384 2564 flags.go:64] FLAG: --log-json-info-buffer-size="0" May 27 21:28:46.847253 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843389 2564 flags.go:64] FLAG: --log-json-split-stream="false" May 27 21:28:46.847253 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843392 2564 flags.go:64] FLAG: --log-text-info-buffer-size="0" May 27 21:28:46.847253 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843395 2564 flags.go:64] FLAG: --log-text-split-stream="false" May 27 21:28:46.847253 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843398 2564 flags.go:64] FLAG: --logging-format="text" May 27 21:28:46.847253 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843401 2564 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" May 27 21:28:46.847253 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843404 2564 flags.go:64] FLAG: --make-iptables-util-chains="true" May 27 21:28:46.847253 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843406 2564 flags.go:64] FLAG: --manifest-url="" May 27 21:28:46.847253 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843409 2564 flags.go:64] FLAG: --manifest-url-header="" May 27 21:28:46.847253 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843418 2564 flags.go:64] FLAG: --max-housekeeping-interval="15s" May 27 21:28:46.847253 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843421 2564 flags.go:64] FLAG: --max-open-files="1000000" May 27 21:28:46.847253 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843425 2564 flags.go:64] FLAG: --max-pods="110" May 27 21:28:46.847253 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843428 2564 flags.go:64] FLAG: --maximum-dead-containers="-1" May 27 21:28:46.847253 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843431 2564 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" May 27 21:28:46.847253 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843434 2564 flags.go:64] FLAG: --memory-manager-policy="None" May 27 21:28:46.847253 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843437 2564 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" May 27 21:28:46.847253 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843440 2564 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" May 27 21:28:46.847253 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843443 2564 flags.go:64] FLAG: --node-ip="0.0.0.0" May 27 21:28:46.847253 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843447 2564 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" May 27 21:28:46.847253 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843463 2564 flags.go:64] FLAG: --node-status-max-images="50" May 27 21:28:46.847253 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843466 2564 flags.go:64] FLAG: --node-status-update-frequency="10s" May 27 21:28:46.847253 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843470 2564 flags.go:64] FLAG: --oom-score-adj="-999" May 27 21:28:46.847253 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843472 2564 flags.go:64] FLAG: --pod-cidr="" May 27 21:28:46.847253 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843475 2564 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:afe88671bf163c42b467d159dc6f0b784dbe28b1ed44e171edc9c06fd99cc6fc" May 27 21:28:46.847818 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843481 2564 flags.go:64] FLAG: --pod-manifest-path="" May 27 21:28:46.847818 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843484 2564 flags.go:64] FLAG: --pod-max-pids="-1" May 27 21:28:46.847818 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843489 2564 flags.go:64] FLAG: --pods-per-core="0" May 27 21:28:46.847818 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843492 2564 flags.go:64] FLAG: --port="10250" May 27 21:28:46.847818 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843495 2564 flags.go:64] FLAG: --protect-kernel-defaults="false" May 27 21:28:46.847818 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843498 2564 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-08388676e4103438c" May 27 21:28:46.847818 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843502 2564 flags.go:64] FLAG: --qos-reserved="" May 27 21:28:46.847818 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843504 2564 flags.go:64] FLAG: --read-only-port="10255" May 27 21:28:46.847818 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843507 2564 flags.go:64] FLAG: --register-node="true" May 27 21:28:46.847818 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843510 2564 flags.go:64] FLAG: --register-schedulable="true" May 27 21:28:46.847818 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843513 2564 flags.go:64] FLAG: --register-with-taints="" May 27 21:28:46.847818 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843516 2564 flags.go:64] FLAG: --registry-burst="10" May 27 21:28:46.847818 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843519 2564 flags.go:64] FLAG: --registry-qps="5" May 27 21:28:46.847818 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843523 2564 flags.go:64] FLAG: --reserved-cpus="" May 27 21:28:46.847818 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843525 2564 flags.go:64] FLAG: --reserved-memory="" May 27 21:28:46.847818 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843530 2564 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" May 27 21:28:46.847818 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843532 2564 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" May 27 21:28:46.847818 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843535 2564 flags.go:64] FLAG: --rotate-certificates="false" May 27 21:28:46.847818 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843538 2564 flags.go:64] FLAG: --rotate-server-certificates="false" May 27 21:28:46.847818 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843541 2564 flags.go:64] FLAG: --runonce="false" May 27 21:28:46.847818 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843543 2564 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" May 27 21:28:46.847818 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843546 2564 flags.go:64] FLAG: --runtime-request-timeout="2m0s" May 27 21:28:46.847818 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843549 2564 flags.go:64] FLAG: --seccomp-default="false" May 27 21:28:46.847818 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843552 2564 flags.go:64] FLAG: --serialize-image-pulls="true" May 27 21:28:46.847818 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843555 2564 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" May 27 21:28:46.847818 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843560 2564 flags.go:64] FLAG: --storage-driver-db="cadvisor" May 27 21:28:46.848456 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843563 2564 flags.go:64] FLAG: --storage-driver-host="localhost:8086" May 27 21:28:46.848456 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843567 2564 flags.go:64] FLAG: --storage-driver-password="root" May 27 21:28:46.848456 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843570 2564 flags.go:64] FLAG: --storage-driver-secure="false" May 27 21:28:46.848456 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843573 2564 flags.go:64] FLAG: --storage-driver-table="stats" May 27 21:28:46.848456 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843575 2564 flags.go:64] FLAG: --storage-driver-user="root" May 27 21:28:46.848456 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843578 2564 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" May 27 21:28:46.848456 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843581 2564 flags.go:64] FLAG: --sync-frequency="1m0s" May 27 21:28:46.848456 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843584 2564 flags.go:64] FLAG: --system-cgroups="" May 27 21:28:46.848456 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843588 2564 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" May 27 21:28:46.848456 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843594 2564 flags.go:64] FLAG: --system-reserved-cgroup="" May 27 21:28:46.848456 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843597 2564 flags.go:64] FLAG: --tls-cert-file="" May 27 21:28:46.848456 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843599 2564 flags.go:64] FLAG: --tls-cipher-suites="[]" May 27 21:28:46.848456 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843603 2564 flags.go:64] FLAG: --tls-min-version="" May 27 21:28:46.848456 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843606 2564 flags.go:64] FLAG: --tls-private-key-file="" May 27 21:28:46.848456 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843609 2564 flags.go:64] FLAG: --topology-manager-policy="none" May 27 21:28:46.848456 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843612 2564 flags.go:64] FLAG: --topology-manager-policy-options="" May 27 21:28:46.848456 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843615 2564 flags.go:64] FLAG: --topology-manager-scope="container" May 27 21:28:46.848456 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843618 2564 flags.go:64] FLAG: --v="2" May 27 21:28:46.848456 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843622 2564 flags.go:64] FLAG: --version="false" May 27 21:28:46.848456 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843626 2564 flags.go:64] FLAG: --vmodule="" May 27 21:28:46.848456 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843630 2564 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" May 27 21:28:46.848456 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.843633 2564 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" May 27 21:28:46.848456 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843726 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 27 21:28:46.848456 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843729 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 27 21:28:46.849024 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843733 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 27 21:28:46.849024 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843736 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 27 21:28:46.849024 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843739 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 27 21:28:46.849024 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843742 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 27 21:28:46.849024 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843746 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 27 21:28:46.849024 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843748 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 27 21:28:46.849024 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843751 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 27 21:28:46.849024 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843755 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 27 21:28:46.849024 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843757 2564 feature_gate.go:328] unrecognized feature gate: Example2 May 27 21:28:46.849024 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843760 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 27 21:28:46.849024 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843763 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 27 21:28:46.849024 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843766 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 27 21:28:46.849024 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843768 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 27 21:28:46.849024 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843770 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 27 21:28:46.849024 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843773 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 27 21:28:46.849024 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843775 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 27 21:28:46.849024 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843780 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 27 21:28:46.849024 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843782 2564 feature_gate.go:328] unrecognized feature gate: NewOLM May 27 21:28:46.849024 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843785 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 27 21:28:46.849024 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843787 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 27 21:28:46.849581 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843790 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 27 21:28:46.849581 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843792 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 27 21:28:46.849581 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843795 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 27 21:28:46.849581 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843797 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 27 21:28:46.849581 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843799 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 27 21:28:46.849581 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843802 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 27 21:28:46.849581 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843804 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 27 21:28:46.849581 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843807 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 27 21:28:46.849581 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843810 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 27 21:28:46.849581 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843812 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 27 21:28:46.849581 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843815 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 27 21:28:46.849581 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843817 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 27 21:28:46.849581 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843820 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 27 21:28:46.849581 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843822 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 27 21:28:46.849581 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843825 2564 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 27 21:28:46.849581 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843827 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 27 21:28:46.849581 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843830 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 27 21:28:46.849581 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843832 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 27 21:28:46.849581 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843835 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 27 21:28:46.849581 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843839 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 27 21:28:46.850088 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843841 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 27 21:28:46.850088 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843844 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability May 27 21:28:46.850088 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843846 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 27 21:28:46.850088 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843850 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 27 21:28:46.850088 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843854 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 27 21:28:46.850088 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843857 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 27 21:28:46.850088 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843860 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 27 21:28:46.850088 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843863 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 27 21:28:46.850088 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843866 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 27 21:28:46.850088 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843869 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 27 21:28:46.850088 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843873 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 27 21:28:46.850088 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843877 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 27 21:28:46.850088 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843879 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 27 21:28:46.850088 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843882 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 27 21:28:46.850088 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843885 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 27 21:28:46.850088 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843888 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 27 21:28:46.850088 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843890 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 27 21:28:46.850088 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843893 2564 feature_gate.go:328] unrecognized feature gate: DualReplica May 27 21:28:46.850088 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843896 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 27 21:28:46.850558 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843898 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 27 21:28:46.850558 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843901 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 27 21:28:46.850558 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843903 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 27 21:28:46.850558 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843905 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 27 21:28:46.850558 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843908 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 27 21:28:46.850558 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843910 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 27 21:28:46.850558 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843912 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 27 21:28:46.850558 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843915 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 27 21:28:46.850558 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843917 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 27 21:28:46.850558 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843920 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 27 21:28:46.850558 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843922 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 27 21:28:46.850558 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843925 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 27 21:28:46.850558 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843928 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 27 21:28:46.850558 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843931 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 27 21:28:46.850558 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843933 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 27 21:28:46.850558 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843936 2564 feature_gate.go:328] unrecognized feature gate: Example May 27 21:28:46.850558 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843939 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 27 21:28:46.850558 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843942 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 27 21:28:46.850558 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843944 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 27 21:28:46.850558 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843947 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 27 21:28:46.851047 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843949 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 27 21:28:46.851047 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843952 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 27 21:28:46.851047 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843955 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 27 21:28:46.851047 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843957 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores May 27 21:28:46.851047 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.843960 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages May 27 21:28:46.851047 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.844470 2564 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} May 27 21:28:46.854279 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.854241 2564 server.go:530] "Kubelet version" kubeletVersion="v1.33.11" May 27 21:28:46.854279 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.854276 2564 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" May 27 21:28:46.854420 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854326 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 27 21:28:46.854420 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854333 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 27 21:28:46.854420 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854336 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 27 21:28:46.854420 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854339 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 27 21:28:46.854420 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854342 2564 feature_gate.go:328] unrecognized feature gate: Example May 27 21:28:46.854420 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854345 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 27 21:28:46.854420 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854348 2564 feature_gate.go:328] unrecognized feature gate: Example2 May 27 21:28:46.854420 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854351 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 27 21:28:46.854420 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854354 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 27 21:28:46.854420 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854358 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 27 21:28:46.854420 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854363 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 27 21:28:46.854420 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854366 2564 feature_gate.go:328] unrecognized feature gate: DualReplica May 27 21:28:46.854420 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854368 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 27 21:28:46.854420 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854371 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 27 21:28:46.854420 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854374 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 27 21:28:46.854420 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854377 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 27 21:28:46.854420 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854380 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 27 21:28:46.854420 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854383 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 27 21:28:46.854420 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854385 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 27 21:28:46.854420 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854388 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 27 21:28:46.854916 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854391 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 27 21:28:46.854916 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854394 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 27 21:28:46.854916 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854397 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 27 21:28:46.854916 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854399 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 27 21:28:46.854916 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854402 2564 feature_gate.go:328] unrecognized feature gate: NewOLM May 27 21:28:46.854916 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854404 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 27 21:28:46.854916 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854407 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 27 21:28:46.854916 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854409 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 27 21:28:46.854916 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854412 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 27 21:28:46.854916 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854414 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 27 21:28:46.854916 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854418 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 27 21:28:46.854916 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854420 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 27 21:28:46.854916 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854423 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 27 21:28:46.854916 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854426 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 27 21:28:46.854916 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854429 2564 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 27 21:28:46.854916 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854432 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 27 21:28:46.854916 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854434 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 27 21:28:46.854916 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854438 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 27 21:28:46.854916 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854441 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 27 21:28:46.854916 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854443 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 27 21:28:46.855432 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854446 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages May 27 21:28:46.855432 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854448 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 27 21:28:46.855432 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854451 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 27 21:28:46.855432 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854455 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 27 21:28:46.855432 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854459 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 27 21:28:46.855432 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854462 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 27 21:28:46.855432 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854464 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 27 21:28:46.855432 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854467 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 27 21:28:46.855432 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854469 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores May 27 21:28:46.855432 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854472 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 27 21:28:46.855432 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854474 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 27 21:28:46.855432 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854477 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 27 21:28:46.855432 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854479 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 27 21:28:46.855432 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854482 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 27 21:28:46.855432 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854484 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 27 21:28:46.855432 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854487 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 27 21:28:46.855432 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854489 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 27 21:28:46.855432 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854492 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 27 21:28:46.855432 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854494 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 27 21:28:46.855432 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854497 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 27 21:28:46.855926 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854499 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 27 21:28:46.855926 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854503 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 27 21:28:46.855926 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854505 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 27 21:28:46.855926 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854508 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 27 21:28:46.855926 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854511 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 27 21:28:46.855926 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854514 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 27 21:28:46.855926 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854516 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability May 27 21:28:46.855926 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854519 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 27 21:28:46.855926 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854522 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 27 21:28:46.855926 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854524 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 27 21:28:46.855926 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854527 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 27 21:28:46.855926 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854530 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 27 21:28:46.855926 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854532 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 27 21:28:46.855926 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854535 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 27 21:28:46.855926 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854537 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 27 21:28:46.855926 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854540 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 27 21:28:46.855926 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854543 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 27 21:28:46.855926 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854545 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 27 21:28:46.855926 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854550 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 27 21:28:46.856442 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854553 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 27 21:28:46.856442 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854557 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 27 21:28:46.856442 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854561 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 27 21:28:46.856442 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854564 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 27 21:28:46.856442 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854567 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 27 21:28:46.856442 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854569 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 27 21:28:46.856442 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854572 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 27 21:28:46.856442 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.854577 2564 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} May 27 21:28:46.856442 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854671 2564 feature_gate.go:328] unrecognized feature gate: Example May 27 21:28:46.856442 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854677 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 27 21:28:46.856442 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854679 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 27 21:28:46.856442 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854682 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages May 27 21:28:46.856442 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854684 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 27 21:28:46.856442 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854687 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 27 21:28:46.856442 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854689 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 27 21:28:46.856442 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854692 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 27 21:28:46.856832 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854695 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 27 21:28:46.856832 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854698 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 27 21:28:46.856832 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854701 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 27 21:28:46.856832 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854704 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 27 21:28:46.856832 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854706 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 27 21:28:46.856832 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854709 2564 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 27 21:28:46.856832 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854711 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 27 21:28:46.856832 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854714 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 27 21:28:46.856832 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854716 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 27 21:28:46.856832 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854719 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 27 21:28:46.856832 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854722 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 27 21:28:46.856832 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854724 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 27 21:28:46.856832 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854727 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 27 21:28:46.856832 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854729 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 27 21:28:46.856832 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854732 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability May 27 21:28:46.856832 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854735 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 27 21:28:46.856832 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854738 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 27 21:28:46.856832 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854741 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 27 21:28:46.856832 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854745 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 27 21:28:46.857329 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854748 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 27 21:28:46.857329 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854750 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 27 21:28:46.857329 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854753 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 27 21:28:46.857329 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854755 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 27 21:28:46.857329 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854757 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 27 21:28:46.857329 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854760 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 27 21:28:46.857329 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854762 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 27 21:28:46.857329 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854765 2564 feature_gate.go:328] unrecognized feature gate: NewOLM May 27 21:28:46.857329 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854767 2564 feature_gate.go:328] unrecognized feature gate: DualReplica May 27 21:28:46.857329 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854769 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 27 21:28:46.857329 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854772 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 27 21:28:46.857329 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854774 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 27 21:28:46.857329 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854777 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 27 21:28:46.857329 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854781 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 27 21:28:46.857329 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854784 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 27 21:28:46.857329 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854788 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 27 21:28:46.857329 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854790 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 27 21:28:46.857329 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854793 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 27 21:28:46.857329 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854795 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores May 27 21:28:46.857329 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854798 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 27 21:28:46.857808 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854801 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 27 21:28:46.857808 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854803 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 27 21:28:46.857808 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854806 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 27 21:28:46.857808 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854808 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 27 21:28:46.857808 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854811 2564 feature_gate.go:328] unrecognized feature gate: Example2 May 27 21:28:46.857808 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854813 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 27 21:28:46.857808 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854815 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 27 21:28:46.857808 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854818 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 27 21:28:46.857808 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854821 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 27 21:28:46.857808 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854824 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 27 21:28:46.857808 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854827 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 27 21:28:46.857808 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854829 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 27 21:28:46.857808 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854831 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 27 21:28:46.857808 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854834 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 27 21:28:46.857808 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854836 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 27 21:28:46.857808 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854839 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 27 21:28:46.857808 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854841 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 27 21:28:46.857808 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854844 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 27 21:28:46.857808 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854846 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 27 21:28:46.857808 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854849 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 27 21:28:46.858303 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854851 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 27 21:28:46.858303 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854854 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 27 21:28:46.858303 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854856 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 27 21:28:46.858303 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854859 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 27 21:28:46.858303 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854861 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 27 21:28:46.858303 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854864 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 27 21:28:46.858303 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854867 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 27 21:28:46.858303 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854869 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 27 21:28:46.858303 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854872 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 27 21:28:46.858303 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854875 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 27 21:28:46.858303 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854877 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 27 21:28:46.858303 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854880 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 27 21:28:46.858303 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854882 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 27 21:28:46.858303 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854884 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 27 21:28:46.858303 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854887 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 27 21:28:46.858303 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854889 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 27 21:28:46.858303 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854892 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 27 21:28:46.858303 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854894 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 27 21:28:46.858303 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:46.854897 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 27 21:28:46.858760 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.854902 2564 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} May 27 21:28:46.858760 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.855649 2564 server.go:962] "Client rotation is on, will bootstrap in background" May 27 21:28:46.860950 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.860936 2564 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" May 27 21:28:46.861878 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.861867 2564 server.go:1019] "Starting client certificate rotation" May 27 21:28:46.861982 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.861963 2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" May 27 21:28:46.862014 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.862004 2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" May 27 21:28:46.884816 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.884794 2564 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" May 27 21:28:46.888632 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.888616 2564 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" May 27 21:28:46.904899 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.904871 2564 log.go:25] "Validated CRI v1 runtime API" May 27 21:28:46.911244 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.911224 2564 log.go:25] "Validated CRI v1 image API" May 27 21:28:46.912708 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.912692 2564 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" May 27 21:28:46.915384 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.915365 2564 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" May 27 21:28:46.915495 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.915432 2564 fs.go:135] Filesystem UUIDs: map[514c868e-185b-4916-822c-cf5fbb9f3cd9:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 b7e0d985-5d34-4671-ab8e-5e9b8e4693f9:/dev/nvme0n1p3] May 27 21:28:46.915495 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.915462 2564 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] May 27 21:28:46.920080 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.919969 2564 manager.go:217] Machine: {Timestamp:2026-05-27 21:28:46.919480719 +0000 UTC m=+0.393685439 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3200434 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f6aa96ea6605c408f6297abecdb63 SystemUUID:ec2f6aa9-6ea6-605c-408f-6297abecdb63 BootID:ee8b0377-edce-4e34-b441-a2535ca9136d Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:f4:a2:6f:26:9b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:f4:a2:6f:26:9b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:a2:9b:4c:93:85:63 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} May 27 21:28:46.920601 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.920589 2564 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. May 27 21:28:46.920697 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.920679 2564 manager.go:233] Version: {KernelVersion:5.14.0-570.114.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260512-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} May 27 21:28:46.923457 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.923421 2564 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] May 27 21:28:46.923599 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.923459 2564 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-246.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} May 27 21:28:46.923641 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.923607 2564 topology_manager.go:138] "Creating topology manager with none policy" May 27 21:28:46.923641 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.923614 2564 container_manager_linux.go:306] "Creating device plugin manager" May 27 21:28:46.923641 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.923622 2564 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" May 27 21:28:46.924453 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.924442 2564 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" May 27 21:28:46.926733 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.926720 2564 state_mem.go:36] "Initialized new in-memory state store" May 27 21:28:46.926858 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.926849 2564 server.go:1267] "Using root directory" path="/var/lib/kubelet" May 27 21:28:46.929956 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.929946 2564 kubelet.go:491] "Attempting to sync node with API server" May 27 21:28:46.929988 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.929961 2564 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests" May 27 21:28:46.929988 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.929977 2564 file.go:69] "Watching path" path="/etc/kubernetes/manifests" May 27 21:28:46.929988 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.929987 2564 kubelet.go:397] "Adding apiserver pod source" May 27 21:28:46.930072 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.929996 2564 apiserver.go:42] "Waiting for node sync before watching apiserver pods" May 27 21:28:46.931484 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.931471 2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" May 27 21:28:46.931525 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.931491 2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" May 27 21:28:46.934451 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.934437 2564 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.11-2.rhaos4.20.gitb2a8320.el9" apiVersion="v1" May 27 21:28:46.935235 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.935202 2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5k285" May 27 21:28:46.936909 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.936896 2564 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" May 27 21:28:46.937823 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.937812 2564 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" May 27 21:28:46.937862 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.937829 2564 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" May 27 21:28:46.937862 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.937835 2564 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" May 27 21:28:46.937862 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.937840 2564 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path" May 27 21:28:46.937862 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.937857 2564 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs" May 27 21:28:46.937862 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.937863 2564 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret" May 27 21:28:46.937985 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.937870 2564 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" May 27 21:28:46.937985 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.937875 2564 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" May 27 21:28:46.937985 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.937882 2564 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc" May 27 21:28:46.937985 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.937888 2564 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap" May 27 21:28:46.937985 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.937900 2564 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected" May 27 21:28:46.938301 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.938292 2564 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" May 27 21:28:46.939971 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.939960 2564 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi" May 27 21:28:46.939971 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.939971 2564 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image" May 27 21:28:46.941850 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:46.941828 2564 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-246.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" May 27 21:28:46.941850 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:46.941834 2564 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" May 27 21:28:46.942839 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.942822 2564 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-246.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope May 27 21:28:46.943161 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.943150 2564 watchdog_linux.go:99] "Systemd watchdog is not enabled" May 27 21:28:46.943199 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.943184 2564 server.go:1295] "Started kubelet" May 27 21:28:46.943278 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.943245 2564 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 May 27 21:28:46.943768 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.943723 2564 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 May 27 21:28:46.943831 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.943794 2564 server_v1.go:47] "podresources" method="list" useActivePods=true May 27 21:28:46.944160 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.944138 2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5k285" May 27 21:28:46.944543 ip-10-0-130-246 systemd[1]: Started Kubernetes Kubelet. May 27 21:28:46.945298 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.945285 2564 server.go:317] "Adding debug handlers to kubelet server" May 27 21:28:46.945791 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.945777 2564 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" May 27 21:28:46.951609 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:46.951587 2564 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" May 27 21:28:46.953586 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.953571 2564 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" May 27 21:28:46.953586 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.953581 2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" May 27 21:28:46.953693 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.953678 2564 volume_manager.go:295] "The desired_state_of_world populator starts" May 27 21:28:46.953693 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.953688 2564 volume_manager.go:297] "Starting Kubelet Volume Manager" May 27 21:28:46.955740 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.955506 2564 desired_state_of_world_populator.go:150] "Desired state populator starts to run" May 27 21:28:46.955975 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.955933 2564 reconstruct.go:97] "Volume reconstruction finished" May 27 21:28:46.956089 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.956080 2564 reconciler.go:26] "Reconciler: start to sync state" May 27 21:28:46.956302 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:46.956287 2564 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-130-246.ec2.internal\" not found" May 27 21:28:46.957886 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.957851 2564 factory.go:55] Registering systemd factory May 27 21:28:46.957973 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.957898 2564 factory.go:223] Registration of the systemd container factory successfully May 27 21:28:46.958336 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.958296 2564 factory.go:153] Registering CRI-O factory May 27 21:28:46.958398 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.958343 2564 factory.go:223] Registration of the crio container factory successfully May 27 21:28:46.958630 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.958615 2564 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory May 27 21:28:46.958672 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.958648 2564 factory.go:103] Registering Raw factory May 27 21:28:46.958672 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.958670 2564 manager.go:1196] Started watching for new ooms in manager May 27 21:28:46.959302 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.959287 2564 manager.go:319] Starting recovery of all containers May 27 21:28:46.960384 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.960363 2564 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" May 27 21:28:46.963380 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:46.963359 2564 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-130-246.ec2.internal\" not found" node="ip-10-0-130-246.ec2.internal" May 27 21:28:46.967678 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.967543 2564 manager.go:324] Recovery completed May 27 21:28:46.971948 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.971936 2564 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 27 21:28:46.973568 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.973554 2564 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-246.ec2.internal" event="NodeHasSufficientMemory" May 27 21:28:46.973645 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.973584 2564 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-246.ec2.internal" event="NodeHasNoDiskPressure" May 27 21:28:46.973645 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.973598 2564 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-246.ec2.internal" event="NodeHasSufficientPID" May 27 21:28:46.974033 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.974022 2564 cpu_manager.go:222] "Starting CPU manager" policy="none" May 27 21:28:46.974097 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.974033 2564 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" May 27 21:28:46.974097 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.974052 2564 state_mem.go:36] "Initialized new in-memory state store" May 27 21:28:46.976774 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.976761 2564 policy_none.go:49] "None policy: Start" May 27 21:28:46.976838 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.976779 2564 memory_manager.go:186] "Starting memorymanager" policy="None" May 27 21:28:46.976838 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:46.976792 2564 state_mem.go:35] "Initializing new in-memory state store" May 27 21:28:47.013837 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.013821 2564 manager.go:341] "Starting Device Plugin manager" May 27 21:28:47.020813 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:47.013861 2564 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" May 27 21:28:47.020813 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.013875 2564 server.go:85] "Starting device plugin registration server" May 27 21:28:47.020813 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.014103 2564 eviction_manager.go:189] "Eviction manager: starting control loop" May 27 21:28:47.020813 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.014114 2564 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" May 27 21:28:47.020813 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.014230 2564 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" May 27 21:28:47.020813 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.014308 2564 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" May 27 21:28:47.020813 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.014316 2564 plugin_manager.go:118] "Starting Kubelet Plugin Manager" May 27 21:28:47.020813 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:47.014692 2564 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" May 27 21:28:47.020813 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:47.014727 2564 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-246.ec2.internal\" not found" May 27 21:28:47.077667 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.077628 2564 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4" May 27 21:28:47.078814 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.078798 2564 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6" May 27 21:28:47.078888 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.078821 2564 status_manager.go:230] "Starting to sync pod status with apiserver" May 27 21:28:47.078888 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.078837 2564 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." May 27 21:28:47.078888 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.078844 2564 kubelet.go:2451] "Starting kubelet main sync loop" May 27 21:28:47.078888 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:47.078879 2564 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" May 27 21:28:47.082370 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.082351 2564 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" May 27 21:28:47.114564 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.114484 2564 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 27 21:28:47.115358 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.115341 2564 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-246.ec2.internal" event="NodeHasSufficientMemory" May 27 21:28:47.115435 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.115374 2564 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-246.ec2.internal" event="NodeHasNoDiskPressure" May 27 21:28:47.115435 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.115389 2564 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-246.ec2.internal" event="NodeHasSufficientPID" May 27 21:28:47.115435 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.115418 2564 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-246.ec2.internal" May 27 21:28:47.124306 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.124290 2564 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-246.ec2.internal" May 27 21:28:47.124350 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:47.124312 2564 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-246.ec2.internal\": node \"ip-10-0-130-246.ec2.internal\" not found" May 27 21:28:47.143093 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:47.143071 2564 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-130-246.ec2.internal\" not found" May 27 21:28:47.179615 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.179547 2564 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-246.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-246.ec2.internal"] May 27 21:28:47.179787 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.179656 2564 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 27 21:28:47.181297 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.181280 2564 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-246.ec2.internal" event="NodeHasSufficientMemory" May 27 21:28:47.181398 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.181308 2564 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-246.ec2.internal" event="NodeHasNoDiskPressure" May 27 21:28:47.181398 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.181320 2564 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-246.ec2.internal" event="NodeHasSufficientPID" May 27 21:28:47.182624 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.182613 2564 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 27 21:28:47.182817 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.182790 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-246.ec2.internal" May 27 21:28:47.182869 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.182822 2564 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 27 21:28:47.183318 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.183303 2564 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-246.ec2.internal" event="NodeHasSufficientMemory" May 27 21:28:47.183420 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.183323 2564 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-246.ec2.internal" event="NodeHasSufficientMemory" May 27 21:28:47.183420 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.183348 2564 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-246.ec2.internal" event="NodeHasNoDiskPressure" May 27 21:28:47.183420 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.183358 2564 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-246.ec2.internal" event="NodeHasSufficientPID" May 27 21:28:47.183420 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.183327 2564 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-246.ec2.internal" event="NodeHasNoDiskPressure" May 27 21:28:47.183420 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.183407 2564 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-246.ec2.internal" event="NodeHasSufficientPID" May 27 21:28:47.184521 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.184508 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-246.ec2.internal" May 27 21:28:47.184573 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.184531 2564 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 27 21:28:47.185131 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.185119 2564 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-246.ec2.internal" event="NodeHasSufficientMemory" May 27 21:28:47.185183 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.185141 2564 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-246.ec2.internal" event="NodeHasNoDiskPressure" May 27 21:28:47.185183 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.185154 2564 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-246.ec2.internal" event="NodeHasSufficientPID" May 27 21:28:47.209087 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:47.209058 2564 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-246.ec2.internal\" not found" node="ip-10-0-130-246.ec2.internal" May 27 21:28:47.213846 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:47.213827 2564 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-246.ec2.internal\" not found" node="ip-10-0-130-246.ec2.internal" May 27 21:28:47.243144 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:47.243118 2564 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-130-246.ec2.internal\" not found" May 27 21:28:47.258520 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.258486 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7e5e8096305be124754f9f48db3e823f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-246.ec2.internal\" (UID: \"7e5e8096305be124754f9f48db3e823f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-246.ec2.internal" May 27 21:28:47.258609 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.258525 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7e5e8096305be124754f9f48db3e823f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-246.ec2.internal\" (UID: \"7e5e8096305be124754f9f48db3e823f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-246.ec2.internal" May 27 21:28:47.258609 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.258546 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6614026242b9bed0ac8a71fe40323c9f-config\") pod \"kube-apiserver-proxy-ip-10-0-130-246.ec2.internal\" (UID: \"6614026242b9bed0ac8a71fe40323c9f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-246.ec2.internal" May 27 21:28:47.343620 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:47.343579 2564 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-130-246.ec2.internal\" not found" May 27 21:28:47.358992 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.358961 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7e5e8096305be124754f9f48db3e823f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-246.ec2.internal\" (UID: \"7e5e8096305be124754f9f48db3e823f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-246.ec2.internal" May 27 21:28:47.359127 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.358917 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7e5e8096305be124754f9f48db3e823f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-246.ec2.internal\" (UID: \"7e5e8096305be124754f9f48db3e823f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-246.ec2.internal" May 27 21:28:47.359127 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.359031 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6614026242b9bed0ac8a71fe40323c9f-config\") pod \"kube-apiserver-proxy-ip-10-0-130-246.ec2.internal\" (UID: \"6614026242b9bed0ac8a71fe40323c9f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-246.ec2.internal" May 27 21:28:47.359127 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.359052 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7e5e8096305be124754f9f48db3e823f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-246.ec2.internal\" (UID: \"7e5e8096305be124754f9f48db3e823f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-246.ec2.internal" May 27 21:28:47.359127 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.359099 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7e5e8096305be124754f9f48db3e823f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-246.ec2.internal\" (UID: \"7e5e8096305be124754f9f48db3e823f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-246.ec2.internal" May 27 21:28:47.359127 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.359099 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6614026242b9bed0ac8a71fe40323c9f-config\") pod \"kube-apiserver-proxy-ip-10-0-130-246.ec2.internal\" (UID: \"6614026242b9bed0ac8a71fe40323c9f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-246.ec2.internal" May 27 21:28:47.444445 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:47.444366 2564 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-130-246.ec2.internal\" not found" May 27 21:28:47.512877 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.512847 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-246.ec2.internal" May 27 21:28:47.517653 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.517617 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-246.ec2.internal" May 27 21:28:47.545236 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:47.545201 2564 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-130-246.ec2.internal\" not found" May 27 21:28:47.645685 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:47.645634 2564 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-130-246.ec2.internal\" not found" May 27 21:28:47.746233 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:47.746158 2564 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-130-246.ec2.internal\" not found" May 27 21:28:47.846678 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:47.846645 2564 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-130-246.ec2.internal\" not found" May 27 21:28:47.862064 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.862037 2564 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" May 27 21:28:47.862224 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.862154 2564 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" May 27 21:28:47.862224 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.862156 2564 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" May 27 21:28:47.905454 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.905428 2564 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" May 27 21:28:47.930800 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.930773 2564 apiserver.go:52] "Watching apiserver" May 27 21:28:47.940624 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.940599 2564 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" May 27 21:28:47.940971 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.940951 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-mdd25","openshift-multus/multus-additional-cni-plugins-4k8dw","openshift-multus/network-metrics-daemon-gxt5q","openshift-network-operator/iptables-alerter-h4fkz","openshift-ovn-kubernetes/ovnkube-node-8p7kk","kube-system/konnectivity-agent-p6vj8","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-knn5k","openshift-cluster-node-tuning-operator/tuned-nls7n","openshift-dns/node-resolver-sc6tb","openshift-multus/multus-sm52n","openshift-network-diagnostics/network-check-target-2kzbj"] May 27 21:28:47.942720 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.942704 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mdd25" May 27 21:28:47.943810 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.943788 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4k8dw" May 27 21:28:47.945067 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.945054 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gxt5q" May 27 21:28:47.945144 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:47.945129 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gxt5q" podUID="7fc93ab3-b601-4fe7-b073-b8702a09404d" May 27 21:28:47.945465 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.945449 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-khf67\"" May 27 21:28:47.946152 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.946141 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-h4fkz" May 27 21:28:47.946524 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.946506 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" May 27 21:28:47.946617 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.946600 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" May 27 21:28:47.946617 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.946612 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" May 27 21:28:47.946730 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.946715 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" May 27 21:28:47.946780 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.946728 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" May 27 21:28:47.947005 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.946988 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" May 27 21:28:47.947105 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.947048 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-s2l9m\"" May 27 21:28:47.947105 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.947073 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" May 27 21:28:47.947220 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.947105 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" May 27 21:28:47.947534 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.947519 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:47.947811 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.947786 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-05-26 21:23:46 +0000 UTC" deadline="2028-02-19 10:23:01.36381228 +0000 UTC" May 27 21:28:47.947811 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.947810 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15180h54m13.41600407s" May 27 21:28:47.948536 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.948521 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-88j95\"" May 27 21:28:47.948616 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.948542 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" May 27 21:28:47.948616 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.948569 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" May 27 21:28:47.948787 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.948774 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" May 27 21:28:47.949180 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.949168 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-p6vj8" May 27 21:28:47.950573 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.950550 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" May 27 21:28:47.951661 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.950817 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" May 27 21:28:47.951661 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.951160 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-knn5k" May 27 21:28:47.952099 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.952080 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" May 27 21:28:47.952371 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.952355 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" May 27 21:28:47.952434 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.952382 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" May 27 21:28:47.952692 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.952671 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" May 27 21:28:47.952792 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.952777 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-jtb6t\"" May 27 21:28:47.953041 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.953024 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-nls7n" May 27 21:28:47.953146 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.953028 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" May 27 21:28:47.953146 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.953114 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-t752l\"" May 27 21:28:47.953247 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.953178 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" May 27 21:28:47.953739 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.953723 2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" May 27 21:28:47.954728 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.954712 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" May 27 21:28:47.954940 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.954926 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-4gsxg\"" May 27 21:28:47.955017 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.954987 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-sc6tb" May 27 21:28:47.955606 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.955592 2564 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-246.ec2.internal" May 27 21:28:47.956479 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.956465 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-sm52n" May 27 21:28:47.956559 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.956498 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" May 27 21:28:47.956988 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.956974 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" May 27 21:28:47.957116 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.957101 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" May 27 21:28:47.957407 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.957393 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" May 27 21:28:47.957750 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.957737 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-c4hqf\"" May 27 21:28:47.957829 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.957814 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2kzbj" May 27 21:28:47.957881 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:47.957863 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2kzbj" podUID="6dd8b333-153e-4494-a210-8d7ab405d008" May 27 21:28:47.958940 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.958926 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-rbkkl\"" May 27 21:28:47.959090 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.959076 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" May 27 21:28:47.959166 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.959153 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" May 27 21:28:47.959238 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.959223 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-4frb9\"" May 27 21:28:47.960913 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.960899 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" May 27 21:28:47.962142 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.962123 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a-host-var-lib-kubelet\") pod \"multus-sm52n\" (UID: \"e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a\") " pod="openshift-multus/multus-sm52n" May 27 21:28:47.962231 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.962145 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/74672ea1-1457-40d1-b8e1-5acca042ca3e-system-cni-dir\") pod \"multus-additional-cni-plugins-4k8dw\" (UID: \"74672ea1-1457-40d1-b8e1-5acca042ca3e\") " pod="openshift-multus/multus-additional-cni-plugins-4k8dw" May 27 21:28:47.962231 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.962201 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9hdg\" (UniqueName: \"kubernetes.io/projected/74672ea1-1457-40d1-b8e1-5acca042ca3e-kube-api-access-g9hdg\") pod \"multus-additional-cni-plugins-4k8dw\" (UID: \"74672ea1-1457-40d1-b8e1-5acca042ca3e\") " pod="openshift-multus/multus-additional-cni-plugins-4k8dw" May 27 21:28:47.962368 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.962234 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5-var-lib-openvswitch\") pod \"ovnkube-node-8p7kk\" (UID: \"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:47.962368 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.962283 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8p7kk\" (UID: \"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:47.962368 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.962310 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/faad2ffc-1823-4376-9f3a-c208b39219d9-var-lib-kubelet\") pod \"tuned-nls7n\" (UID: \"faad2ffc-1823-4376-9f3a-c208b39219d9\") " pod="openshift-cluster-node-tuning-operator/tuned-nls7n" May 27 21:28:47.962368 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.962334 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5-log-socket\") pod \"ovnkube-node-8p7kk\" (UID: \"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:47.962368 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.962357 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbzzk\" (UniqueName: \"kubernetes.io/projected/fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5-kube-api-access-qbzzk\") pod \"ovnkube-node-8p7kk\" (UID: \"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:47.962854 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.962381 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a-cni-binary-copy\") pod \"multus-sm52n\" (UID: \"e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a\") " pod="openshift-multus/multus-sm52n" May 27 21:28:47.962854 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.962406 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a-etc-kubernetes\") pod \"multus-sm52n\" (UID: \"e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a\") " pod="openshift-multus/multus-sm52n" May 27 21:28:47.962854 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.962432 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bc955a1c-63ba-4921-8127-a0f27ae757e3-etc-selinux\") pod \"aws-ebs-csi-driver-node-knn5k\" (UID: \"bc955a1c-63ba-4921-8127-a0f27ae757e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-knn5k" May 27 21:28:47.962854 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.962451 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/faad2ffc-1823-4376-9f3a-c208b39219d9-etc-modprobe-d\") pod \"tuned-nls7n\" (UID: \"faad2ffc-1823-4376-9f3a-c208b39219d9\") " pod="openshift-cluster-node-tuning-operator/tuned-nls7n" May 27 21:28:47.962854 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.962466 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/faad2ffc-1823-4376-9f3a-c208b39219d9-lib-modules\") pod \"tuned-nls7n\" (UID: \"faad2ffc-1823-4376-9f3a-c208b39219d9\") " pod="openshift-cluster-node-tuning-operator/tuned-nls7n" May 27 21:28:47.962854 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.962480 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7fc93ab3-b601-4fe7-b073-b8702a09404d-metrics-certs\") pod \"network-metrics-daemon-gxt5q\" (UID: \"7fc93ab3-b601-4fe7-b073-b8702a09404d\") " pod="openshift-multus/network-metrics-daemon-gxt5q" May 27 21:28:47.962854 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.962513 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5-host-kubelet\") pod \"ovnkube-node-8p7kk\" (UID: \"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:47.962854 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.962537 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mx9gp\" (UniqueName: \"kubernetes.io/projected/d895055d-d1d4-44a6-9024-8b244e769d5e-kube-api-access-mx9gp\") pod \"node-resolver-sc6tb\" (UID: \"d895055d-d1d4-44a6-9024-8b244e769d5e\") " pod="openshift-dns/node-resolver-sc6tb" May 27 21:28:47.962854 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.962557 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e176311e-f128-4137-904b-7dd6fea5eed1-host\") pod \"node-ca-mdd25\" (UID: \"e176311e-f128-4137-904b-7dd6fea5eed1\") " pod="openshift-image-registry/node-ca-mdd25" May 27 21:28:47.962854 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.962579 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a-host-run-netns\") pod \"multus-sm52n\" (UID: \"e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a\") " pod="openshift-multus/multus-sm52n" May 27 21:28:47.962854 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.962606 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a-host-run-multus-certs\") pod \"multus-sm52n\" (UID: \"e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a\") " pod="openshift-multus/multus-sm52n" May 27 21:28:47.962854 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.962670 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/74672ea1-1457-40d1-b8e1-5acca042ca3e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4k8dw\" (UID: \"74672ea1-1457-40d1-b8e1-5acca042ca3e\") " pod="openshift-multus/multus-additional-cni-plugins-4k8dw" May 27 21:28:47.962854 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.962703 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/657348d2-3465-4003-9ef6-3139dc8c1a92-konnectivity-ca\") pod \"konnectivity-agent-p6vj8\" (UID: \"657348d2-3465-4003-9ef6-3139dc8c1a92\") " pod="kube-system/konnectivity-agent-p6vj8" May 27 21:28:47.962854 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.962735 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/faad2ffc-1823-4376-9f3a-c208b39219d9-etc-kubernetes\") pod \"tuned-nls7n\" (UID: \"faad2ffc-1823-4376-9f3a-c208b39219d9\") " pod="openshift-cluster-node-tuning-operator/tuned-nls7n" May 27 21:28:47.962854 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.962758 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/faad2ffc-1823-4376-9f3a-c208b39219d9-etc-systemd\") pod \"tuned-nls7n\" (UID: \"faad2ffc-1823-4376-9f3a-c208b39219d9\") " pod="openshift-cluster-node-tuning-operator/tuned-nls7n" May 27 21:28:47.962854 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.962782 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a-os-release\") pod \"multus-sm52n\" (UID: \"e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a\") " pod="openshift-multus/multus-sm52n" May 27 21:28:47.962854 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.962806 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a-multus-daemon-config\") pod \"multus-sm52n\" (UID: \"e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a\") " pod="openshift-multus/multus-sm52n" May 27 21:28:47.963569 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.962860 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/74672ea1-1457-40d1-b8e1-5acca042ca3e-cni-binary-copy\") pod \"multus-additional-cni-plugins-4k8dw\" (UID: \"74672ea1-1457-40d1-b8e1-5acca042ca3e\") " pod="openshift-multus/multus-additional-cni-plugins-4k8dw" May 27 21:28:47.963569 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.962881 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/faad2ffc-1823-4376-9f3a-c208b39219d9-run\") pod \"tuned-nls7n\" (UID: \"faad2ffc-1823-4376-9f3a-c208b39219d9\") " pod="openshift-cluster-node-tuning-operator/tuned-nls7n" May 27 21:28:47.963569 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.962898 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/faad2ffc-1823-4376-9f3a-c208b39219d9-host\") pod \"tuned-nls7n\" (UID: \"faad2ffc-1823-4376-9f3a-c208b39219d9\") " pod="openshift-cluster-node-tuning-operator/tuned-nls7n" May 27 21:28:47.963569 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.962921 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a-host-var-lib-cni-bin\") pod \"multus-sm52n\" (UID: \"e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a\") " pod="openshift-multus/multus-sm52n" May 27 21:28:47.963569 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.962936 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a-hostroot\") pod \"multus-sm52n\" (UID: \"e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a\") " pod="openshift-multus/multus-sm52n" May 27 21:28:47.963569 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.962971 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgsgw\" (UniqueName: \"kubernetes.io/projected/e176311e-f128-4137-904b-7dd6fea5eed1-kube-api-access-qgsgw\") pod \"node-ca-mdd25\" (UID: \"e176311e-f128-4137-904b-7dd6fea5eed1\") " pod="openshift-image-registry/node-ca-mdd25" May 27 21:28:47.963569 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.962997 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bc955a1c-63ba-4921-8127-a0f27ae757e3-socket-dir\") pod \"aws-ebs-csi-driver-node-knn5k\" (UID: \"bc955a1c-63ba-4921-8127-a0f27ae757e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-knn5k" May 27 21:28:47.963569 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.963019 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5-node-log\") pod \"ovnkube-node-8p7kk\" (UID: \"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:47.963569 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.963054 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d895055d-d1d4-44a6-9024-8b244e769d5e-hosts-file\") pod \"node-resolver-sc6tb\" (UID: \"d895055d-d1d4-44a6-9024-8b244e769d5e\") " pod="openshift-dns/node-resolver-sc6tb" May 27 21:28:47.963569 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.963071 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/74672ea1-1457-40d1-b8e1-5acca042ca3e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4k8dw\" (UID: \"74672ea1-1457-40d1-b8e1-5acca042ca3e\") " pod="openshift-multus/multus-additional-cni-plugins-4k8dw" May 27 21:28:47.963569 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.963087 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bc955a1c-63ba-4921-8127-a0f27ae757e3-registration-dir\") pod \"aws-ebs-csi-driver-node-knn5k\" (UID: \"bc955a1c-63ba-4921-8127-a0f27ae757e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-knn5k" May 27 21:28:47.963569 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.963109 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/faad2ffc-1823-4376-9f3a-c208b39219d9-etc-sysctl-d\") pod \"tuned-nls7n\" (UID: \"faad2ffc-1823-4376-9f3a-c208b39219d9\") " pod="openshift-cluster-node-tuning-operator/tuned-nls7n" May 27 21:28:47.963569 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.963127 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5-ovn-node-metrics-cert\") pod \"ovnkube-node-8p7kk\" (UID: \"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:47.963569 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.963141 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d895055d-d1d4-44a6-9024-8b244e769d5e-tmp-dir\") pod \"node-resolver-sc6tb\" (UID: \"d895055d-d1d4-44a6-9024-8b244e769d5e\") " pod="openshift-dns/node-resolver-sc6tb" May 27 21:28:47.963569 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.963177 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a-system-cni-dir\") pod \"multus-sm52n\" (UID: \"e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a\") " pod="openshift-multus/multus-sm52n" May 27 21:28:47.963569 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.963206 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5-host-cni-bin\") pod \"ovnkube-node-8p7kk\" (UID: \"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:47.963569 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.963234 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e176311e-f128-4137-904b-7dd6fea5eed1-serviceca\") pod \"node-ca-mdd25\" (UID: \"e176311e-f128-4137-904b-7dd6fea5eed1\") " pod="openshift-image-registry/node-ca-mdd25" May 27 21:28:47.964168 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.963258 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c33b5127-3c83-44e2-a1e0-5c0e433c9f00-iptables-alerter-script\") pod \"iptables-alerter-h4fkz\" (UID: \"c33b5127-3c83-44e2-a1e0-5c0e433c9f00\") " pod="openshift-network-operator/iptables-alerter-h4fkz" May 27 21:28:47.964168 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.963307 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bc955a1c-63ba-4921-8127-a0f27ae757e3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-knn5k\" (UID: \"bc955a1c-63ba-4921-8127-a0f27ae757e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-knn5k" May 27 21:28:47.964168 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.963328 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/74672ea1-1457-40d1-b8e1-5acca042ca3e-cnibin\") pod \"multus-additional-cni-plugins-4k8dw\" (UID: \"74672ea1-1457-40d1-b8e1-5acca042ca3e\") " pod="openshift-multus/multus-additional-cni-plugins-4k8dw" May 27 21:28:47.964168 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.963343 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/74672ea1-1457-40d1-b8e1-5acca042ca3e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4k8dw\" (UID: \"74672ea1-1457-40d1-b8e1-5acca042ca3e\") " pod="openshift-multus/multus-additional-cni-plugins-4k8dw" May 27 21:28:47.964168 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.963360 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/faad2ffc-1823-4376-9f3a-c208b39219d9-etc-sysctl-conf\") pod \"tuned-nls7n\" (UID: \"faad2ffc-1823-4376-9f3a-c208b39219d9\") " pod="openshift-cluster-node-tuning-operator/tuned-nls7n" May 27 21:28:47.964168 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.963374 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/faad2ffc-1823-4376-9f3a-c208b39219d9-etc-tuned\") pod \"tuned-nls7n\" (UID: \"faad2ffc-1823-4376-9f3a-c208b39219d9\") " pod="openshift-cluster-node-tuning-operator/tuned-nls7n" May 27 21:28:47.964168 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.963388 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5-etc-openvswitch\") pod \"ovnkube-node-8p7kk\" (UID: \"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:47.964168 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.963402 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5-host-run-ovn-kubernetes\") pod \"ovnkube-node-8p7kk\" (UID: \"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:47.964168 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.963415 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5-env-overrides\") pod \"ovnkube-node-8p7kk\" (UID: \"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:47.964168 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.963429 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a-multus-conf-dir\") pod \"multus-sm52n\" (UID: \"e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a\") " pod="openshift-multus/multus-sm52n" May 27 21:28:47.964168 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.963466 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/faad2ffc-1823-4376-9f3a-c208b39219d9-etc-sysconfig\") pod \"tuned-nls7n\" (UID: \"faad2ffc-1823-4376-9f3a-c208b39219d9\") " pod="openshift-cluster-node-tuning-operator/tuned-nls7n" May 27 21:28:47.964168 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.963492 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/faad2ffc-1823-4376-9f3a-c208b39219d9-tmp\") pod \"tuned-nls7n\" (UID: \"faad2ffc-1823-4376-9f3a-c208b39219d9\") " pod="openshift-cluster-node-tuning-operator/tuned-nls7n" May 27 21:28:47.964168 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.963524 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5-ovnkube-config\") pod \"ovnkube-node-8p7kk\" (UID: \"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:47.964168 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.963551 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a-multus-socket-dir-parent\") pod \"multus-sm52n\" (UID: \"e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a\") " pod="openshift-multus/multus-sm52n" May 27 21:28:47.964168 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.963585 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a-host-run-k8s-cni-cncf-io\") pod \"multus-sm52n\" (UID: \"e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a\") " pod="openshift-multus/multus-sm52n" May 27 21:28:47.964168 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.963610 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pb7pn\" (UniqueName: \"kubernetes.io/projected/e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a-kube-api-access-pb7pn\") pod \"multus-sm52n\" (UID: \"e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a\") " pod="openshift-multus/multus-sm52n" May 27 21:28:47.964605 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.963631 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/faad2ffc-1823-4376-9f3a-c208b39219d9-sys\") pod \"tuned-nls7n\" (UID: \"faad2ffc-1823-4376-9f3a-c208b39219d9\") " pod="openshift-cluster-node-tuning-operator/tuned-nls7n" May 27 21:28:47.964605 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.963655 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5-systemd-units\") pod \"ovnkube-node-8p7kk\" (UID: \"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:47.964605 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.963687 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a-multus-cni-dir\") pod \"multus-sm52n\" (UID: \"e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a\") " pod="openshift-multus/multus-sm52n" May 27 21:28:47.964605 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.963701 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a-cnibin\") pod \"multus-sm52n\" (UID: \"e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a\") " pod="openshift-multus/multus-sm52n" May 27 21:28:47.964605 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.963731 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bc955a1c-63ba-4921-8127-a0f27ae757e3-sys-fs\") pod \"aws-ebs-csi-driver-node-knn5k\" (UID: \"bc955a1c-63ba-4921-8127-a0f27ae757e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-knn5k" May 27 21:28:47.964605 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.963755 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8s8n6\" (UniqueName: \"kubernetes.io/projected/faad2ffc-1823-4376-9f3a-c208b39219d9-kube-api-access-8s8n6\") pod \"tuned-nls7n\" (UID: \"faad2ffc-1823-4376-9f3a-c208b39219d9\") " pod="openshift-cluster-node-tuning-operator/tuned-nls7n" May 27 21:28:47.964605 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.963769 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5-host-cni-netd\") pod \"ovnkube-node-8p7kk\" (UID: \"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:47.964605 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.963782 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a-host-var-lib-cni-multus\") pod \"multus-sm52n\" (UID: \"e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a\") " pod="openshift-multus/multus-sm52n" May 27 21:28:47.964605 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.963795 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/74672ea1-1457-40d1-b8e1-5acca042ca3e-os-release\") pod \"multus-additional-cni-plugins-4k8dw\" (UID: \"74672ea1-1457-40d1-b8e1-5acca042ca3e\") " pod="openshift-multus/multus-additional-cni-plugins-4k8dw" May 27 21:28:47.964605 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.963809 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c33b5127-3c83-44e2-a1e0-5c0e433c9f00-host-slash\") pod \"iptables-alerter-h4fkz\" (UID: \"c33b5127-3c83-44e2-a1e0-5c0e433c9f00\") " pod="openshift-network-operator/iptables-alerter-h4fkz" May 27 21:28:47.964605 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.963822 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/657348d2-3465-4003-9ef6-3139dc8c1a92-agent-certs\") pod \"konnectivity-agent-p6vj8\" (UID: \"657348d2-3465-4003-9ef6-3139dc8c1a92\") " pod="kube-system/konnectivity-agent-p6vj8" May 27 21:28:47.964605 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.963836 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4294l\" (UniqueName: \"kubernetes.io/projected/bc955a1c-63ba-4921-8127-a0f27ae757e3-kube-api-access-4294l\") pod \"aws-ebs-csi-driver-node-knn5k\" (UID: \"bc955a1c-63ba-4921-8127-a0f27ae757e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-knn5k" May 27 21:28:47.964605 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.963850 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmrbd\" (UniqueName: \"kubernetes.io/projected/7fc93ab3-b601-4fe7-b073-b8702a09404d-kube-api-access-rmrbd\") pod \"network-metrics-daemon-gxt5q\" (UID: \"7fc93ab3-b601-4fe7-b073-b8702a09404d\") " pod="openshift-multus/network-metrics-daemon-gxt5q" May 27 21:28:47.964605 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.963877 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5-run-systemd\") pod \"ovnkube-node-8p7kk\" (UID: \"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:47.964605 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.963904 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5-ovnkube-script-lib\") pod \"ovnkube-node-8p7kk\" (UID: \"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:47.964605 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.963926 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5-run-ovn\") pod \"ovnkube-node-8p7kk\" (UID: \"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:47.965018 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.963953 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8khm\" (UniqueName: \"kubernetes.io/projected/c33b5127-3c83-44e2-a1e0-5c0e433c9f00-kube-api-access-v8khm\") pod \"iptables-alerter-h4fkz\" (UID: \"c33b5127-3c83-44e2-a1e0-5c0e433c9f00\") " pod="openshift-network-operator/iptables-alerter-h4fkz" May 27 21:28:47.965018 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.963988 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bc955a1c-63ba-4921-8127-a0f27ae757e3-device-dir\") pod \"aws-ebs-csi-driver-node-knn5k\" (UID: \"bc955a1c-63ba-4921-8127-a0f27ae757e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-knn5k" May 27 21:28:47.965018 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.964010 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5-host-slash\") pod \"ovnkube-node-8p7kk\" (UID: \"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:47.965018 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.964034 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5-host-run-netns\") pod \"ovnkube-node-8p7kk\" (UID: \"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:47.965018 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.964057 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5-run-openvswitch\") pod \"ovnkube-node-8p7kk\" (UID: \"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:47.970219 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.970202 2564 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" May 27 21:28:47.975144 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.975130 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-246.ec2.internal"] May 27 21:28:47.976924 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.976912 2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" May 27 21:28:47.976978 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.976969 2564 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-246.ec2.internal" May 27 21:28:47.984661 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.984646 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-130-246.ec2.internal"] May 27 21:28:47.984774 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.984762 2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" May 27 21:28:47.998456 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:47.998440 2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-jb887" May 27 21:28:48.008329 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.008305 2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-jb887" May 27 21:28:48.057382 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.057364 2564 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world" May 27 21:28:48.064692 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.064671 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/faad2ffc-1823-4376-9f3a-c208b39219d9-var-lib-kubelet\") pod \"tuned-nls7n\" (UID: \"faad2ffc-1823-4376-9f3a-c208b39219d9\") " pod="openshift-cluster-node-tuning-operator/tuned-nls7n" May 27 21:28:48.064692 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.064698 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5-log-socket\") pod \"ovnkube-node-8p7kk\" (UID: \"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:48.064963 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.064722 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qbzzk\" (UniqueName: \"kubernetes.io/projected/fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5-kube-api-access-qbzzk\") pod \"ovnkube-node-8p7kk\" (UID: \"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:48.064963 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.064737 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a-cni-binary-copy\") pod \"multus-sm52n\" (UID: \"e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a\") " pod="openshift-multus/multus-sm52n" May 27 21:28:48.064963 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.064751 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a-etc-kubernetes\") pod \"multus-sm52n\" (UID: \"e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a\") " pod="openshift-multus/multus-sm52n" May 27 21:28:48.064963 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.064771 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bc955a1c-63ba-4921-8127-a0f27ae757e3-etc-selinux\") pod \"aws-ebs-csi-driver-node-knn5k\" (UID: \"bc955a1c-63ba-4921-8127-a0f27ae757e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-knn5k" May 27 21:28:48.064963 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.064782 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5-log-socket\") pod \"ovnkube-node-8p7kk\" (UID: \"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:48.064963 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.064793 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/faad2ffc-1823-4376-9f3a-c208b39219d9-etc-modprobe-d\") pod \"tuned-nls7n\" (UID: \"faad2ffc-1823-4376-9f3a-c208b39219d9\") " pod="openshift-cluster-node-tuning-operator/tuned-nls7n" May 27 21:28:48.064963 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.064782 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/faad2ffc-1823-4376-9f3a-c208b39219d9-var-lib-kubelet\") pod \"tuned-nls7n\" (UID: \"faad2ffc-1823-4376-9f3a-c208b39219d9\") " pod="openshift-cluster-node-tuning-operator/tuned-nls7n" May 27 21:28:48.064963 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.064816 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/faad2ffc-1823-4376-9f3a-c208b39219d9-lib-modules\") pod \"tuned-nls7n\" (UID: \"faad2ffc-1823-4376-9f3a-c208b39219d9\") " pod="openshift-cluster-node-tuning-operator/tuned-nls7n" May 27 21:28:48.064963 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.064840 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7fc93ab3-b601-4fe7-b073-b8702a09404d-metrics-certs\") pod \"network-metrics-daemon-gxt5q\" (UID: \"7fc93ab3-b601-4fe7-b073-b8702a09404d\") " pod="openshift-multus/network-metrics-daemon-gxt5q" May 27 21:28:48.064963 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.064856 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a-etc-kubernetes\") pod \"multus-sm52n\" (UID: \"e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a\") " pod="openshift-multus/multus-sm52n" May 27 21:28:48.064963 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.064862 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5-host-kubelet\") pod \"ovnkube-node-8p7kk\" (UID: \"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:48.064963 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.064874 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bc955a1c-63ba-4921-8127-a0f27ae757e3-etc-selinux\") pod \"aws-ebs-csi-driver-node-knn5k\" (UID: \"bc955a1c-63ba-4921-8127-a0f27ae757e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-knn5k" May 27 21:28:48.064963 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.064887 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/faad2ffc-1823-4376-9f3a-c208b39219d9-etc-modprobe-d\") pod \"tuned-nls7n\" (UID: \"faad2ffc-1823-4376-9f3a-c208b39219d9\") " pod="openshift-cluster-node-tuning-operator/tuned-nls7n" May 27 21:28:48.064963 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.064894 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5-host-kubelet\") pod \"ovnkube-node-8p7kk\" (UID: \"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:48.064963 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.064895 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mx9gp\" (UniqueName: \"kubernetes.io/projected/d895055d-d1d4-44a6-9024-8b244e769d5e-kube-api-access-mx9gp\") pod \"node-resolver-sc6tb\" (UID: \"d895055d-d1d4-44a6-9024-8b244e769d5e\") " pod="openshift-dns/node-resolver-sc6tb" May 27 21:28:48.064963 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.064944 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e176311e-f128-4137-904b-7dd6fea5eed1-host\") pod \"node-ca-mdd25\" (UID: \"e176311e-f128-4137-904b-7dd6fea5eed1\") " pod="openshift-image-registry/node-ca-mdd25" May 27 21:28:48.064963 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.064961 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/faad2ffc-1823-4376-9f3a-c208b39219d9-lib-modules\") pod \"tuned-nls7n\" (UID: \"faad2ffc-1823-4376-9f3a-c208b39219d9\") " pod="openshift-cluster-node-tuning-operator/tuned-nls7n" May 27 21:28:48.064963 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.064967 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a-host-run-netns\") pod \"multus-sm52n\" (UID: \"e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a\") " pod="openshift-multus/multus-sm52n" May 27 21:28:48.065894 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:48.064973 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 27 21:28:48.065894 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.064992 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a-host-run-multus-certs\") pod \"multus-sm52n\" (UID: \"e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a\") " pod="openshift-multus/multus-sm52n" May 27 21:28:48.065894 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.065021 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/74672ea1-1457-40d1-b8e1-5acca042ca3e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4k8dw\" (UID: \"74672ea1-1457-40d1-b8e1-5acca042ca3e\") " pod="openshift-multus/multus-additional-cni-plugins-4k8dw" May 27 21:28:48.065894 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:48.065051 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fc93ab3-b601-4fe7-b073-b8702a09404d-metrics-certs podName:7fc93ab3-b601-4fe7-b073-b8702a09404d nodeName:}" failed. No retries permitted until 2026-05-27 21:28:48.565021982 +0000 UTC m=+2.039226698 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7fc93ab3-b601-4fe7-b073-b8702a09404d-metrics-certs") pod "network-metrics-daemon-gxt5q" (UID: "7fc93ab3-b601-4fe7-b073-b8702a09404d") : object "openshift-multus"/"metrics-daemon-secret" not registered May 27 21:28:48.065894 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.065082 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/657348d2-3465-4003-9ef6-3139dc8c1a92-konnectivity-ca\") pod \"konnectivity-agent-p6vj8\" (UID: \"657348d2-3465-4003-9ef6-3139dc8c1a92\") " pod="kube-system/konnectivity-agent-p6vj8" May 27 21:28:48.065894 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.065086 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a-host-run-netns\") pod \"multus-sm52n\" (UID: \"e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a\") " pod="openshift-multus/multus-sm52n" May 27 21:28:48.065894 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.065106 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/faad2ffc-1823-4376-9f3a-c208b39219d9-etc-kubernetes\") pod \"tuned-nls7n\" (UID: \"faad2ffc-1823-4376-9f3a-c208b39219d9\") " pod="openshift-cluster-node-tuning-operator/tuned-nls7n" May 27 21:28:48.065894 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.065106 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a-host-run-multus-certs\") pod \"multus-sm52n\" (UID: \"e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a\") " pod="openshift-multus/multus-sm52n" May 27 21:28:48.065894 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.065128 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/faad2ffc-1823-4376-9f3a-c208b39219d9-etc-systemd\") pod \"tuned-nls7n\" (UID: \"faad2ffc-1823-4376-9f3a-c208b39219d9\") " pod="openshift-cluster-node-tuning-operator/tuned-nls7n" May 27 21:28:48.065894 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.065137 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e176311e-f128-4137-904b-7dd6fea5eed1-host\") pod \"node-ca-mdd25\" (UID: \"e176311e-f128-4137-904b-7dd6fea5eed1\") " pod="openshift-image-registry/node-ca-mdd25" May 27 21:28:48.065894 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.065447 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a-os-release\") pod \"multus-sm52n\" (UID: \"e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a\") " pod="openshift-multus/multus-sm52n" May 27 21:28:48.065894 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.065495 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a-multus-daemon-config\") pod \"multus-sm52n\" (UID: \"e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a\") " pod="openshift-multus/multus-sm52n" May 27 21:28:48.065894 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.065525 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/74672ea1-1457-40d1-b8e1-5acca042ca3e-cni-binary-copy\") pod \"multus-additional-cni-plugins-4k8dw\" (UID: \"74672ea1-1457-40d1-b8e1-5acca042ca3e\") " pod="openshift-multus/multus-additional-cni-plugins-4k8dw" May 27 21:28:48.065894 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.065335 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/faad2ffc-1823-4376-9f3a-c208b39219d9-etc-kubernetes\") pod \"tuned-nls7n\" (UID: \"faad2ffc-1823-4376-9f3a-c208b39219d9\") " pod="openshift-cluster-node-tuning-operator/tuned-nls7n" May 27 21:28:48.065894 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.065557 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/faad2ffc-1823-4376-9f3a-c208b39219d9-run\") pod \"tuned-nls7n\" (UID: \"faad2ffc-1823-4376-9f3a-c208b39219d9\") " pod="openshift-cluster-node-tuning-operator/tuned-nls7n" May 27 21:28:48.065894 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.065587 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/faad2ffc-1823-4376-9f3a-c208b39219d9-host\") pod \"tuned-nls7n\" (UID: \"faad2ffc-1823-4376-9f3a-c208b39219d9\") " pod="openshift-cluster-node-tuning-operator/tuned-nls7n" May 27 21:28:48.065894 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.065681 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/faad2ffc-1823-4376-9f3a-c208b39219d9-run\") pod \"tuned-nls7n\" (UID: \"faad2ffc-1823-4376-9f3a-c208b39219d9\") " pod="openshift-cluster-node-tuning-operator/tuned-nls7n" May 27 21:28:48.066815 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.065720 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a-host-var-lib-cni-bin\") pod \"multus-sm52n\" (UID: \"e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a\") " pod="openshift-multus/multus-sm52n" May 27 21:28:48.066815 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.065747 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a-hostroot\") pod \"multus-sm52n\" (UID: \"e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a\") " pod="openshift-multus/multus-sm52n" May 27 21:28:48.066815 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.065786 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qgsgw\" (UniqueName: \"kubernetes.io/projected/e176311e-f128-4137-904b-7dd6fea5eed1-kube-api-access-qgsgw\") pod \"node-ca-mdd25\" (UID: \"e176311e-f128-4137-904b-7dd6fea5eed1\") " pod="openshift-image-registry/node-ca-mdd25" May 27 21:28:48.066815 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.065819 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bc955a1c-63ba-4921-8127-a0f27ae757e3-socket-dir\") pod \"aws-ebs-csi-driver-node-knn5k\" (UID: \"bc955a1c-63ba-4921-8127-a0f27ae757e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-knn5k" May 27 21:28:48.066815 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.065888 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a-cni-binary-copy\") pod \"multus-sm52n\" (UID: \"e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a\") " pod="openshift-multus/multus-sm52n" May 27 21:28:48.066815 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.065928 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/74672ea1-1457-40d1-b8e1-5acca042ca3e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4k8dw\" (UID: \"74672ea1-1457-40d1-b8e1-5acca042ca3e\") " pod="openshift-multus/multus-additional-cni-plugins-4k8dw" May 27 21:28:48.066815 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.066028 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5-node-log\") pod \"ovnkube-node-8p7kk\" (UID: \"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:48.066815 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.066073 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d895055d-d1d4-44a6-9024-8b244e769d5e-hosts-file\") pod \"node-resolver-sc6tb\" (UID: \"d895055d-d1d4-44a6-9024-8b244e769d5e\") " pod="openshift-dns/node-resolver-sc6tb" May 27 21:28:48.066815 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.066129 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/74672ea1-1457-40d1-b8e1-5acca042ca3e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4k8dw\" (UID: \"74672ea1-1457-40d1-b8e1-5acca042ca3e\") " pod="openshift-multus/multus-additional-cni-plugins-4k8dw" May 27 21:28:48.066815 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.066174 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bc955a1c-63ba-4921-8127-a0f27ae757e3-registration-dir\") pod \"aws-ebs-csi-driver-node-knn5k\" (UID: \"bc955a1c-63ba-4921-8127-a0f27ae757e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-knn5k" May 27 21:28:48.066815 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.066220 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/faad2ffc-1823-4376-9f3a-c208b39219d9-etc-sysctl-d\") pod \"tuned-nls7n\" (UID: \"faad2ffc-1823-4376-9f3a-c208b39219d9\") " pod="openshift-cluster-node-tuning-operator/tuned-nls7n" May 27 21:28:48.066815 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.066311 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/faad2ffc-1823-4376-9f3a-c208b39219d9-etc-systemd\") pod \"tuned-nls7n\" (UID: \"faad2ffc-1823-4376-9f3a-c208b39219d9\") " pod="openshift-cluster-node-tuning-operator/tuned-nls7n" May 27 21:28:48.066815 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.066174 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/657348d2-3465-4003-9ef6-3139dc8c1a92-konnectivity-ca\") pod \"konnectivity-agent-p6vj8\" (UID: \"657348d2-3465-4003-9ef6-3139dc8c1a92\") " pod="kube-system/konnectivity-agent-p6vj8" May 27 21:28:48.066815 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.066541 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a-host-var-lib-cni-bin\") pod \"multus-sm52n\" (UID: \"e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a\") " pod="openshift-multus/multus-sm52n" May 27 21:28:48.066815 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.066645 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/faad2ffc-1823-4376-9f3a-c208b39219d9-host\") pod \"tuned-nls7n\" (UID: \"faad2ffc-1823-4376-9f3a-c208b39219d9\") " pod="openshift-cluster-node-tuning-operator/tuned-nls7n" May 27 21:28:48.066815 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.066713 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5-ovn-node-metrics-cert\") pod \"ovnkube-node-8p7kk\" (UID: \"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:48.067596 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.066832 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/74672ea1-1457-40d1-b8e1-5acca042ca3e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4k8dw\" (UID: \"74672ea1-1457-40d1-b8e1-5acca042ca3e\") " pod="openshift-multus/multus-additional-cni-plugins-4k8dw" May 27 21:28:48.067596 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.066909 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bc955a1c-63ba-4921-8127-a0f27ae757e3-socket-dir\") pod \"aws-ebs-csi-driver-node-knn5k\" (UID: \"bc955a1c-63ba-4921-8127-a0f27ae757e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-knn5k" May 27 21:28:48.067596 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.066924 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bc955a1c-63ba-4921-8127-a0f27ae757e3-registration-dir\") pod \"aws-ebs-csi-driver-node-knn5k\" (UID: \"bc955a1c-63ba-4921-8127-a0f27ae757e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-knn5k" May 27 21:28:48.067596 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.066963 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/74672ea1-1457-40d1-b8e1-5acca042ca3e-cni-binary-copy\") pod \"multus-additional-cni-plugins-4k8dw\" (UID: \"74672ea1-1457-40d1-b8e1-5acca042ca3e\") " pod="openshift-multus/multus-additional-cni-plugins-4k8dw" May 27 21:28:48.067596 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.066988 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a-os-release\") pod \"multus-sm52n\" (UID: \"e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a\") " pod="openshift-multus/multus-sm52n" May 27 21:28:48.067596 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.067014 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d895055d-d1d4-44a6-9024-8b244e769d5e-tmp-dir\") pod \"node-resolver-sc6tb\" (UID: \"d895055d-d1d4-44a6-9024-8b244e769d5e\") " pod="openshift-dns/node-resolver-sc6tb" May 27 21:28:48.067596 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.067032 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/faad2ffc-1823-4376-9f3a-c208b39219d9-etc-sysctl-d\") pod \"tuned-nls7n\" (UID: \"faad2ffc-1823-4376-9f3a-c208b39219d9\") " pod="openshift-cluster-node-tuning-operator/tuned-nls7n" May 27 21:28:48.067596 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.067049 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a-system-cni-dir\") pod \"multus-sm52n\" (UID: \"e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a\") " pod="openshift-multus/multus-sm52n" May 27 21:28:48.067596 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.067082 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5-host-cni-bin\") pod \"ovnkube-node-8p7kk\" (UID: \"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:48.067596 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.067151 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a-hostroot\") pod \"multus-sm52n\" (UID: \"e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a\") " pod="openshift-multus/multus-sm52n" May 27 21:28:48.067596 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.067162 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e176311e-f128-4137-904b-7dd6fea5eed1-serviceca\") pod \"node-ca-mdd25\" (UID: \"e176311e-f128-4137-904b-7dd6fea5eed1\") " pod="openshift-image-registry/node-ca-mdd25" May 27 21:28:48.067596 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.067406 2564 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" May 27 21:28:48.067596 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.067436 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d895055d-d1d4-44a6-9024-8b244e769d5e-hosts-file\") pod \"node-resolver-sc6tb\" (UID: \"d895055d-d1d4-44a6-9024-8b244e769d5e\") " pod="openshift-dns/node-resolver-sc6tb" May 27 21:28:48.067596 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.067498 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5-host-cni-bin\") pod \"ovnkube-node-8p7kk\" (UID: \"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:48.067596 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.067567 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a-multus-daemon-config\") pod \"multus-sm52n\" (UID: \"e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a\") " pod="openshift-multus/multus-sm52n" May 27 21:28:48.068487 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.067657 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5-node-log\") pod \"ovnkube-node-8p7kk\" (UID: \"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:48.068487 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.067839 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d895055d-d1d4-44a6-9024-8b244e769d5e-tmp-dir\") pod \"node-resolver-sc6tb\" (UID: \"d895055d-d1d4-44a6-9024-8b244e769d5e\") " pod="openshift-dns/node-resolver-sc6tb" May 27 21:28:48.068487 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.067925 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a-system-cni-dir\") pod \"multus-sm52n\" (UID: \"e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a\") " pod="openshift-multus/multus-sm52n" May 27 21:28:48.068487 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.067964 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c33b5127-3c83-44e2-a1e0-5c0e433c9f00-iptables-alerter-script\") pod \"iptables-alerter-h4fkz\" (UID: \"c33b5127-3c83-44e2-a1e0-5c0e433c9f00\") " pod="openshift-network-operator/iptables-alerter-h4fkz" May 27 21:28:48.068487 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.067409 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c33b5127-3c83-44e2-a1e0-5c0e433c9f00-iptables-alerter-script\") pod \"iptables-alerter-h4fkz\" (UID: \"c33b5127-3c83-44e2-a1e0-5c0e433c9f00\") " pod="openshift-network-operator/iptables-alerter-h4fkz" May 27 21:28:48.068487 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.068033 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bc955a1c-63ba-4921-8127-a0f27ae757e3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-knn5k\" (UID: \"bc955a1c-63ba-4921-8127-a0f27ae757e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-knn5k" May 27 21:28:48.068487 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.068055 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e176311e-f128-4137-904b-7dd6fea5eed1-serviceca\") pod \"node-ca-mdd25\" (UID: \"e176311e-f128-4137-904b-7dd6fea5eed1\") " pod="openshift-image-registry/node-ca-mdd25" May 27 21:28:48.068487 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.068073 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/74672ea1-1457-40d1-b8e1-5acca042ca3e-cnibin\") pod \"multus-additional-cni-plugins-4k8dw\" (UID: \"74672ea1-1457-40d1-b8e1-5acca042ca3e\") " pod="openshift-multus/multus-additional-cni-plugins-4k8dw" May 27 21:28:48.068487 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.068125 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/74672ea1-1457-40d1-b8e1-5acca042ca3e-cnibin\") pod \"multus-additional-cni-plugins-4k8dw\" (UID: \"74672ea1-1457-40d1-b8e1-5acca042ca3e\") " pod="openshift-multus/multus-additional-cni-plugins-4k8dw" May 27 21:28:48.068487 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.068193 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/74672ea1-1457-40d1-b8e1-5acca042ca3e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4k8dw\" (UID: \"74672ea1-1457-40d1-b8e1-5acca042ca3e\") " pod="openshift-multus/multus-additional-cni-plugins-4k8dw" May 27 21:28:48.068487 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.068224 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/faad2ffc-1823-4376-9f3a-c208b39219d9-etc-sysctl-conf\") pod \"tuned-nls7n\" (UID: \"faad2ffc-1823-4376-9f3a-c208b39219d9\") " pod="openshift-cluster-node-tuning-operator/tuned-nls7n" May 27 21:28:48.068487 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.068244 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bc955a1c-63ba-4921-8127-a0f27ae757e3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-knn5k\" (UID: \"bc955a1c-63ba-4921-8127-a0f27ae757e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-knn5k" May 27 21:28:48.068487 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.068249 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/faad2ffc-1823-4376-9f3a-c208b39219d9-etc-tuned\") pod \"tuned-nls7n\" (UID: \"faad2ffc-1823-4376-9f3a-c208b39219d9\") " pod="openshift-cluster-node-tuning-operator/tuned-nls7n" May 27 21:28:48.068487 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.068394 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/faad2ffc-1823-4376-9f3a-c208b39219d9-etc-sysctl-conf\") pod \"tuned-nls7n\" (UID: \"faad2ffc-1823-4376-9f3a-c208b39219d9\") " pod="openshift-cluster-node-tuning-operator/tuned-nls7n" May 27 21:28:48.068487 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.068440 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5-etc-openvswitch\") pod \"ovnkube-node-8p7kk\" (UID: \"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:48.068487 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.068479 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5-host-run-ovn-kubernetes\") pod \"ovnkube-node-8p7kk\" (UID: \"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:48.069214 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.068516 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5-env-overrides\") pod \"ovnkube-node-8p7kk\" (UID: \"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:48.069214 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.068550 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a-multus-conf-dir\") pod \"multus-sm52n\" (UID: \"e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a\") " pod="openshift-multus/multus-sm52n" May 27 21:28:48.069214 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.068585 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/faad2ffc-1823-4376-9f3a-c208b39219d9-etc-sysconfig\") pod \"tuned-nls7n\" (UID: \"faad2ffc-1823-4376-9f3a-c208b39219d9\") " pod="openshift-cluster-node-tuning-operator/tuned-nls7n" May 27 21:28:48.069214 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.068614 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/faad2ffc-1823-4376-9f3a-c208b39219d9-tmp\") pod \"tuned-nls7n\" (UID: \"faad2ffc-1823-4376-9f3a-c208b39219d9\") " pod="openshift-cluster-node-tuning-operator/tuned-nls7n" May 27 21:28:48.069214 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.068648 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5-ovnkube-config\") pod \"ovnkube-node-8p7kk\" (UID: \"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:48.069214 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.068682 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a-multus-socket-dir-parent\") pod \"multus-sm52n\" (UID: \"e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a\") " pod="openshift-multus/multus-sm52n" May 27 21:28:48.069214 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.068717 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a-host-run-k8s-cni-cncf-io\") pod \"multus-sm52n\" (UID: \"e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a\") " pod="openshift-multus/multus-sm52n" May 27 21:28:48.069214 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.068752 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pb7pn\" (UniqueName: \"kubernetes.io/projected/e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a-kube-api-access-pb7pn\") pod \"multus-sm52n\" (UID: \"e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a\") " pod="openshift-multus/multus-sm52n" May 27 21:28:48.069214 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.068779 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/faad2ffc-1823-4376-9f3a-c208b39219d9-sys\") pod \"tuned-nls7n\" (UID: \"faad2ffc-1823-4376-9f3a-c208b39219d9\") " pod="openshift-cluster-node-tuning-operator/tuned-nls7n" May 27 21:28:48.069214 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.068828 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5-systemd-units\") pod \"ovnkube-node-8p7kk\" (UID: \"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:48.069214 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.068861 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a-multus-cni-dir\") pod \"multus-sm52n\" (UID: \"e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a\") " pod="openshift-multus/multus-sm52n" May 27 21:28:48.069214 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.068894 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a-cnibin\") pod \"multus-sm52n\" (UID: \"e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a\") " pod="openshift-multus/multus-sm52n" May 27 21:28:48.069214 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.068927 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bc955a1c-63ba-4921-8127-a0f27ae757e3-sys-fs\") pod \"aws-ebs-csi-driver-node-knn5k\" (UID: \"bc955a1c-63ba-4921-8127-a0f27ae757e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-knn5k" May 27 21:28:48.069214 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.068961 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8s8n6\" (UniqueName: \"kubernetes.io/projected/faad2ffc-1823-4376-9f3a-c208b39219d9-kube-api-access-8s8n6\") pod \"tuned-nls7n\" (UID: \"faad2ffc-1823-4376-9f3a-c208b39219d9\") " pod="openshift-cluster-node-tuning-operator/tuned-nls7n" May 27 21:28:48.069214 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.068993 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5-host-cni-netd\") pod \"ovnkube-node-8p7kk\" (UID: \"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:48.069214 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.069026 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a-host-var-lib-cni-multus\") pod \"multus-sm52n\" (UID: \"e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a\") " pod="openshift-multus/multus-sm52n" May 27 21:28:48.069214 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.069060 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/74672ea1-1457-40d1-b8e1-5acca042ca3e-os-release\") pod \"multus-additional-cni-plugins-4k8dw\" (UID: \"74672ea1-1457-40d1-b8e1-5acca042ca3e\") " pod="openshift-multus/multus-additional-cni-plugins-4k8dw" May 27 21:28:48.069214 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.069098 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c33b5127-3c83-44e2-a1e0-5c0e433c9f00-host-slash\") pod \"iptables-alerter-h4fkz\" (UID: \"c33b5127-3c83-44e2-a1e0-5c0e433c9f00\") " pod="openshift-network-operator/iptables-alerter-h4fkz" May 27 21:28:48.070673 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.069131 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/657348d2-3465-4003-9ef6-3139dc8c1a92-agent-certs\") pod \"konnectivity-agent-p6vj8\" (UID: \"657348d2-3465-4003-9ef6-3139dc8c1a92\") " pod="kube-system/konnectivity-agent-p6vj8" May 27 21:28:48.070673 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.069160 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4294l\" (UniqueName: \"kubernetes.io/projected/bc955a1c-63ba-4921-8127-a0f27ae757e3-kube-api-access-4294l\") pod \"aws-ebs-csi-driver-node-knn5k\" (UID: \"bc955a1c-63ba-4921-8127-a0f27ae757e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-knn5k" May 27 21:28:48.070673 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.069197 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rmrbd\" (UniqueName: \"kubernetes.io/projected/7fc93ab3-b601-4fe7-b073-b8702a09404d-kube-api-access-rmrbd\") pod \"network-metrics-daemon-gxt5q\" (UID: \"7fc93ab3-b601-4fe7-b073-b8702a09404d\") " pod="openshift-multus/network-metrics-daemon-gxt5q" May 27 21:28:48.070673 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.069236 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5-run-systemd\") pod \"ovnkube-node-8p7kk\" (UID: \"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:48.070673 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.069289 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5-ovnkube-script-lib\") pod \"ovnkube-node-8p7kk\" (UID: \"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:48.070673 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.069330 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5cwl\" (UniqueName: \"kubernetes.io/projected/6dd8b333-153e-4494-a210-8d7ab405d008-kube-api-access-p5cwl\") pod \"network-check-target-2kzbj\" (UID: \"6dd8b333-153e-4494-a210-8d7ab405d008\") " pod="openshift-network-diagnostics/network-check-target-2kzbj" May 27 21:28:48.070673 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.069362 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5-run-ovn\") pod \"ovnkube-node-8p7kk\" (UID: \"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:48.070673 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.069419 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-v8khm\" (UniqueName: \"kubernetes.io/projected/c33b5127-3c83-44e2-a1e0-5c0e433c9f00-kube-api-access-v8khm\") pod \"iptables-alerter-h4fkz\" (UID: \"c33b5127-3c83-44e2-a1e0-5c0e433c9f00\") " pod="openshift-network-operator/iptables-alerter-h4fkz" May 27 21:28:48.070673 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.069476 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bc955a1c-63ba-4921-8127-a0f27ae757e3-device-dir\") pod \"aws-ebs-csi-driver-node-knn5k\" (UID: \"bc955a1c-63ba-4921-8127-a0f27ae757e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-knn5k" May 27 21:28:48.070673 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.069513 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5-host-slash\") pod \"ovnkube-node-8p7kk\" (UID: \"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:48.070673 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.069547 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5-host-run-netns\") pod \"ovnkube-node-8p7kk\" (UID: \"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:48.070673 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.069580 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5-run-openvswitch\") pod \"ovnkube-node-8p7kk\" (UID: \"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:48.070673 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.069614 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a-host-var-lib-kubelet\") pod \"multus-sm52n\" (UID: \"e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a\") " pod="openshift-multus/multus-sm52n" May 27 21:28:48.070673 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.069644 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/74672ea1-1457-40d1-b8e1-5acca042ca3e-system-cni-dir\") pod \"multus-additional-cni-plugins-4k8dw\" (UID: \"74672ea1-1457-40d1-b8e1-5acca042ca3e\") " pod="openshift-multus/multus-additional-cni-plugins-4k8dw" May 27 21:28:48.070673 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.069678 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-g9hdg\" (UniqueName: \"kubernetes.io/projected/74672ea1-1457-40d1-b8e1-5acca042ca3e-kube-api-access-g9hdg\") pod \"multus-additional-cni-plugins-4k8dw\" (UID: \"74672ea1-1457-40d1-b8e1-5acca042ca3e\") " pod="openshift-multus/multus-additional-cni-plugins-4k8dw" May 27 21:28:48.070673 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.069710 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5-var-lib-openvswitch\") pod \"ovnkube-node-8p7kk\" (UID: \"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:48.070673 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.069744 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8p7kk\" (UID: \"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:48.071605 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.069850 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8p7kk\" (UID: \"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:48.071605 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.069872 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/74672ea1-1457-40d1-b8e1-5acca042ca3e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4k8dw\" (UID: \"74672ea1-1457-40d1-b8e1-5acca042ca3e\") " pod="openshift-multus/multus-additional-cni-plugins-4k8dw" May 27 21:28:48.071605 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.069906 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5-etc-openvswitch\") pod \"ovnkube-node-8p7kk\" (UID: \"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:48.071605 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.069944 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/74672ea1-1457-40d1-b8e1-5acca042ca3e-os-release\") pod \"multus-additional-cni-plugins-4k8dw\" (UID: \"74672ea1-1457-40d1-b8e1-5acca042ca3e\") " pod="openshift-multus/multus-additional-cni-plugins-4k8dw" May 27 21:28:48.071605 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.069959 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c33b5127-3c83-44e2-a1e0-5c0e433c9f00-host-slash\") pod \"iptables-alerter-h4fkz\" (UID: \"c33b5127-3c83-44e2-a1e0-5c0e433c9f00\") " pod="openshift-network-operator/iptables-alerter-h4fkz" May 27 21:28:48.071605 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.070028 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/faad2ffc-1823-4376-9f3a-c208b39219d9-sys\") pod \"tuned-nls7n\" (UID: \"faad2ffc-1823-4376-9f3a-c208b39219d9\") " pod="openshift-cluster-node-tuning-operator/tuned-nls7n" May 27 21:28:48.071605 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.070445 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5-host-run-ovn-kubernetes\") pod \"ovnkube-node-8p7kk\" (UID: \"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:48.071605 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.070878 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/faad2ffc-1823-4376-9f3a-c208b39219d9-etc-sysconfig\") pod \"tuned-nls7n\" (UID: \"faad2ffc-1823-4376-9f3a-c208b39219d9\") " pod="openshift-cluster-node-tuning-operator/tuned-nls7n" May 27 21:28:48.071605 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.071095 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5-ovn-node-metrics-cert\") pod \"ovnkube-node-8p7kk\" (UID: \"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:48.071605 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.071115 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5-run-systemd\") pod \"ovnkube-node-8p7kk\" (UID: \"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:48.072203 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.071625 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/faad2ffc-1823-4376-9f3a-c208b39219d9-etc-tuned\") pod \"tuned-nls7n\" (UID: \"faad2ffc-1823-4376-9f3a-c208b39219d9\") " pod="openshift-cluster-node-tuning-operator/tuned-nls7n" May 27 21:28:48.072203 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.071732 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5-ovnkube-script-lib\") pod \"ovnkube-node-8p7kk\" (UID: \"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:48.072203 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.071831 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5-run-ovn\") pod \"ovnkube-node-8p7kk\" (UID: \"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:48.072366 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.072222 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5-env-overrides\") pod \"ovnkube-node-8p7kk\" (UID: \"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:48.072705 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.072685 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bc955a1c-63ba-4921-8127-a0f27ae757e3-device-dir\") pod \"aws-ebs-csi-driver-node-knn5k\" (UID: \"bc955a1c-63ba-4921-8127-a0f27ae757e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-knn5k" May 27 21:28:48.072803 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.072769 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/657348d2-3465-4003-9ef6-3139dc8c1a92-agent-certs\") pod \"konnectivity-agent-p6vj8\" (UID: \"657348d2-3465-4003-9ef6-3139dc8c1a92\") " pod="kube-system/konnectivity-agent-p6vj8" May 27 21:28:48.073148 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.073111 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/74672ea1-1457-40d1-b8e1-5acca042ca3e-system-cni-dir\") pod \"multus-additional-cni-plugins-4k8dw\" (UID: \"74672ea1-1457-40d1-b8e1-5acca042ca3e\") " pod="openshift-multus/multus-additional-cni-plugins-4k8dw" May 27 21:28:48.073234 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.073193 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5-ovnkube-config\") pod \"ovnkube-node-8p7kk\" (UID: \"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:48.073471 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.073450 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5-systemd-units\") pod \"ovnkube-node-8p7kk\" (UID: \"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:48.073605 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.073528 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a-multus-cni-dir\") pod \"multus-sm52n\" (UID: \"e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a\") " pod="openshift-multus/multus-sm52n" May 27 21:28:48.073739 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.073694 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5-var-lib-openvswitch\") pod \"ovnkube-node-8p7kk\" (UID: \"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:48.073827 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.073775 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a-cnibin\") pod \"multus-sm52n\" (UID: \"e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a\") " pod="openshift-multus/multus-sm52n" May 27 21:28:48.073894 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.073834 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a-multus-conf-dir\") pod \"multus-sm52n\" (UID: \"e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a\") " pod="openshift-multus/multus-sm52n" May 27 21:28:48.073948 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.073926 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bc955a1c-63ba-4921-8127-a0f27ae757e3-sys-fs\") pod \"aws-ebs-csi-driver-node-knn5k\" (UID: \"bc955a1c-63ba-4921-8127-a0f27ae757e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-knn5k" May 27 21:28:48.074252 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.074009 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a-multus-socket-dir-parent\") pod \"multus-sm52n\" (UID: \"e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a\") " pod="openshift-multus/multus-sm52n" May 27 21:28:48.074252 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.074025 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/faad2ffc-1823-4376-9f3a-c208b39219d9-tmp\") pod \"tuned-nls7n\" (UID: \"faad2ffc-1823-4376-9f3a-c208b39219d9\") " pod="openshift-cluster-node-tuning-operator/tuned-nls7n" May 27 21:28:48.074252 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.074156 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a-host-run-k8s-cni-cncf-io\") pod \"multus-sm52n\" (UID: \"e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a\") " pod="openshift-multus/multus-sm52n" May 27 21:28:48.074252 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.074209 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5-host-cni-netd\") pod \"ovnkube-node-8p7kk\" (UID: \"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:48.074252 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.074211 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a-host-var-lib-cni-multus\") pod \"multus-sm52n\" (UID: \"e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a\") " pod="openshift-multus/multus-sm52n" May 27 21:28:48.074733 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.074257 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5-host-run-netns\") pod \"ovnkube-node-8p7kk\" (UID: \"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:48.074733 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.074313 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5-run-openvswitch\") pod \"ovnkube-node-8p7kk\" (UID: \"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:48.074733 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.074416 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a-host-var-lib-kubelet\") pod \"multus-sm52n\" (UID: \"e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a\") " pod="openshift-multus/multus-sm52n" May 27 21:28:48.074733 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.074472 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5-host-slash\") pod \"ovnkube-node-8p7kk\" (UID: \"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:48.075668 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.075648 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mx9gp\" (UniqueName: \"kubernetes.io/projected/d895055d-d1d4-44a6-9024-8b244e769d5e-kube-api-access-mx9gp\") pod \"node-resolver-sc6tb\" (UID: \"d895055d-d1d4-44a6-9024-8b244e769d5e\") " pod="openshift-dns/node-resolver-sc6tb" May 27 21:28:48.075823 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.075804 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbzzk\" (UniqueName: \"kubernetes.io/projected/fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5-kube-api-access-qbzzk\") pod \"ovnkube-node-8p7kk\" (UID: \"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:48.076291 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.076257 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgsgw\" (UniqueName: \"kubernetes.io/projected/e176311e-f128-4137-904b-7dd6fea5eed1-kube-api-access-qgsgw\") pod \"node-ca-mdd25\" (UID: \"e176311e-f128-4137-904b-7dd6fea5eed1\") " pod="openshift-image-registry/node-ca-mdd25" May 27 21:28:48.078517 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.078495 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4294l\" (UniqueName: \"kubernetes.io/projected/bc955a1c-63ba-4921-8127-a0f27ae757e3-kube-api-access-4294l\") pod \"aws-ebs-csi-driver-node-knn5k\" (UID: \"bc955a1c-63ba-4921-8127-a0f27ae757e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-knn5k" May 27 21:28:48.078698 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.078670 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmrbd\" (UniqueName: \"kubernetes.io/projected/7fc93ab3-b601-4fe7-b073-b8702a09404d-kube-api-access-rmrbd\") pod \"network-metrics-daemon-gxt5q\" (UID: \"7fc93ab3-b601-4fe7-b073-b8702a09404d\") " pod="openshift-multus/network-metrics-daemon-gxt5q" May 27 21:28:48.079779 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.079760 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pb7pn\" (UniqueName: \"kubernetes.io/projected/e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a-kube-api-access-pb7pn\") pod \"multus-sm52n\" (UID: \"e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a\") " pod="openshift-multus/multus-sm52n" May 27 21:28:48.080393 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.080371 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8khm\" (UniqueName: \"kubernetes.io/projected/c33b5127-3c83-44e2-a1e0-5c0e433c9f00-kube-api-access-v8khm\") pod \"iptables-alerter-h4fkz\" (UID: \"c33b5127-3c83-44e2-a1e0-5c0e433c9f00\") " pod="openshift-network-operator/iptables-alerter-h4fkz" May 27 21:28:48.080902 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.080885 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9hdg\" (UniqueName: \"kubernetes.io/projected/74672ea1-1457-40d1-b8e1-5acca042ca3e-kube-api-access-g9hdg\") pod \"multus-additional-cni-plugins-4k8dw\" (UID: \"74672ea1-1457-40d1-b8e1-5acca042ca3e\") " pod="openshift-multus/multus-additional-cni-plugins-4k8dw" May 27 21:28:48.081553 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.081534 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8s8n6\" (UniqueName: \"kubernetes.io/projected/faad2ffc-1823-4376-9f3a-c208b39219d9-kube-api-access-8s8n6\") pod \"tuned-nls7n\" (UID: \"faad2ffc-1823-4376-9f3a-c208b39219d9\") " pod="openshift-cluster-node-tuning-operator/tuned-nls7n" May 27 21:28:48.098573 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.098553 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-nls7n" May 27 21:28:48.116016 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.115994 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-sc6tb" May 27 21:28:48.121167 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.121150 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-sm52n" May 27 21:28:48.170159 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.170125 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p5cwl\" (UniqueName: \"kubernetes.io/projected/6dd8b333-153e-4494-a210-8d7ab405d008-kube-api-access-p5cwl\") pod \"network-check-target-2kzbj\" (UID: \"6dd8b333-153e-4494-a210-8d7ab405d008\") " pod="openshift-network-diagnostics/network-check-target-2kzbj" May 27 21:28:48.177431 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:48.177404 2564 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 27 21:28:48.177519 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:48.177436 2564 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 27 21:28:48.177519 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:48.177451 2564 projected.go:194] Error preparing data for projected volume kube-api-access-p5cwl for pod openshift-network-diagnostics/network-check-target-2kzbj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 27 21:28:48.177585 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:48.177575 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6dd8b333-153e-4494-a210-8d7ab405d008-kube-api-access-p5cwl podName:6dd8b333-153e-4494-a210-8d7ab405d008 nodeName:}" failed. No retries permitted until 2026-05-27 21:28:48.677552999 +0000 UTC m=+2.151757723 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-p5cwl" (UniqueName: "kubernetes.io/projected/6dd8b333-153e-4494-a210-8d7ab405d008-kube-api-access-p5cwl") pod "network-check-target-2kzbj" (UID: "6dd8b333-153e-4494-a210-8d7ab405d008") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 27 21:28:48.177927 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:48.177903 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e5e8096305be124754f9f48db3e823f.slice/crio-68dcb85c876e789af4f9914d69527dd8a35c9ef4ee6abe697f0d89d527fd9451 WatchSource:0}: Error finding container 68dcb85c876e789af4f9914d69527dd8a35c9ef4ee6abe697f0d89d527fd9451: Status 404 returned error can't find the container with id 68dcb85c876e789af4f9914d69527dd8a35c9ef4ee6abe697f0d89d527fd9451 May 27 21:28:48.178133 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:48.178112 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6614026242b9bed0ac8a71fe40323c9f.slice/crio-9e5ecdf19dd2259bf4c9fc0bf3e97c8397cac55bbf3fb6b778bb2892d2259618 WatchSource:0}: Error finding container 9e5ecdf19dd2259bf4c9fc0bf3e97c8397cac55bbf3fb6b778bb2892d2259618: Status 404 returned error can't find the container with id 9e5ecdf19dd2259bf4c9fc0bf3e97c8397cac55bbf3fb6b778bb2892d2259618 May 27 21:28:48.184336 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.184321 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 27 21:28:48.282760 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.282678 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mdd25" May 27 21:28:48.288983 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:48.288954 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode176311e_f128_4137_904b_7dd6fea5eed1.slice/crio-6cfa50d2c881d44e1100b2c4ba34ce5328b2445e1a7fb32d438ccb631b45db99 WatchSource:0}: Error finding container 6cfa50d2c881d44e1100b2c4ba34ce5328b2445e1a7fb32d438ccb631b45db99: Status 404 returned error can't find the container with id 6cfa50d2c881d44e1100b2c4ba34ce5328b2445e1a7fb32d438ccb631b45db99 May 27 21:28:48.296363 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.296345 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4k8dw" May 27 21:28:48.320788 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.320767 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-h4fkz" May 27 21:28:48.326669 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:48.326637 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc33b5127_3c83_44e2_a1e0_5c0e433c9f00.slice/crio-0b03e93cafc2da8e2fcaf62cd725c0178be4ee8e78704cd6c37d5380a7268876 WatchSource:0}: Error finding container 0b03e93cafc2da8e2fcaf62cd725c0178be4ee8e78704cd6c37d5380a7268876: Status 404 returned error can't find the container with id 0b03e93cafc2da8e2fcaf62cd725c0178be4ee8e78704cd6c37d5380a7268876 May 27 21:28:48.329255 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.329237 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:28:48.335482 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:48.335438 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb58bd7a_94ec_46dc_bc70_5bd3c7a90aa5.slice/crio-4cfeed8fc04c04d48de1ee3d45f0be53e393e41185ed43ccac9a1458a9fef329 WatchSource:0}: Error finding container 4cfeed8fc04c04d48de1ee3d45f0be53e393e41185ed43ccac9a1458a9fef329: Status 404 returned error can't find the container with id 4cfeed8fc04c04d48de1ee3d45f0be53e393e41185ed43ccac9a1458a9fef329 May 27 21:28:48.347698 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.347678 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-p6vj8" May 27 21:28:48.353213 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:48.353190 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod657348d2_3465_4003_9ef6_3139dc8c1a92.slice/crio-0c89cc93a568d0fb76b293fb04c347dc508346a75bbd9e018a6123df007ad7eb WatchSource:0}: Error finding container 0c89cc93a568d0fb76b293fb04c347dc508346a75bbd9e018a6123df007ad7eb: Status 404 returned error can't find the container with id 0c89cc93a568d0fb76b293fb04c347dc508346a75bbd9e018a6123df007ad7eb May 27 21:28:48.354247 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.354230 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-knn5k" May 27 21:28:48.417642 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:48.417607 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1f1d6bc_2c10_482e_bfc1_f5a17bcef82a.slice/crio-626406db0126fc1fb3edbbd19373506e945f1657b5662f7ddc11d0cde9cbdb32 WatchSource:0}: Error finding container 626406db0126fc1fb3edbbd19373506e945f1657b5662f7ddc11d0cde9cbdb32: Status 404 returned error can't find the container with id 626406db0126fc1fb3edbbd19373506e945f1657b5662f7ddc11d0cde9cbdb32 May 27 21:28:48.418345 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:48.418321 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfaad2ffc_1823_4376_9f3a_c208b39219d9.slice/crio-ae191c612bbe40cab269f86dd746f13acd208644f169f642c2f9eac27165466a WatchSource:0}: Error finding container ae191c612bbe40cab269f86dd746f13acd208644f169f642c2f9eac27165466a: Status 404 returned error can't find the container with id ae191c612bbe40cab269f86dd746f13acd208644f169f642c2f9eac27165466a May 27 21:28:48.437550 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:28:48.437524 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd895055d_d1d4_44a6_9024_8b244e769d5e.slice/crio-0b5d94235a2a7bf0a3a83cf5d450e328c5c2ac7068a996e1a404e35c33572d03 WatchSource:0}: Error finding container 0b5d94235a2a7bf0a3a83cf5d450e328c5c2ac7068a996e1a404e35c33572d03: Status 404 returned error can't find the container with id 0b5d94235a2a7bf0a3a83cf5d450e328c5c2ac7068a996e1a404e35c33572d03 May 27 21:28:48.463787 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.463768 2564 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" May 27 21:28:48.572480 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.572395 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7fc93ab3-b601-4fe7-b073-b8702a09404d-metrics-certs\") pod \"network-metrics-daemon-gxt5q\" (UID: \"7fc93ab3-b601-4fe7-b073-b8702a09404d\") " pod="openshift-multus/network-metrics-daemon-gxt5q" May 27 21:28:48.572611 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:48.572507 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 27 21:28:48.572611 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:48.572554 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fc93ab3-b601-4fe7-b073-b8702a09404d-metrics-certs podName:7fc93ab3-b601-4fe7-b073-b8702a09404d nodeName:}" failed. No retries permitted until 2026-05-27 21:28:49.572542244 +0000 UTC m=+3.046746951 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7fc93ab3-b601-4fe7-b073-b8702a09404d-metrics-certs") pod "network-metrics-daemon-gxt5q" (UID: "7fc93ab3-b601-4fe7-b073-b8702a09404d") : object "openshift-multus"/"metrics-daemon-secret" not registered May 27 21:28:48.774525 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.774377 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p5cwl\" (UniqueName: \"kubernetes.io/projected/6dd8b333-153e-4494-a210-8d7ab405d008-kube-api-access-p5cwl\") pod \"network-check-target-2kzbj\" (UID: \"6dd8b333-153e-4494-a210-8d7ab405d008\") " pod="openshift-network-diagnostics/network-check-target-2kzbj" May 27 21:28:48.774694 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:48.774548 2564 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 27 21:28:48.774694 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:48.774573 2564 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 27 21:28:48.774694 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:48.774587 2564 projected.go:194] Error preparing data for projected volume kube-api-access-p5cwl for pod openshift-network-diagnostics/network-check-target-2kzbj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 27 21:28:48.774694 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:48.774649 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6dd8b333-153e-4494-a210-8d7ab405d008-kube-api-access-p5cwl podName:6dd8b333-153e-4494-a210-8d7ab405d008 nodeName:}" failed. No retries permitted until 2026-05-27 21:28:49.774628917 +0000 UTC m=+3.248833629 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-p5cwl" (UniqueName: "kubernetes.io/projected/6dd8b333-153e-4494-a210-8d7ab405d008-kube-api-access-p5cwl") pod "network-check-target-2kzbj" (UID: "6dd8b333-153e-4494-a210-8d7ab405d008") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 27 21:28:48.836764 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:48.836686 2564 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" May 27 21:28:49.010773 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:49.010664 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-26 21:23:47 +0000 UTC" deadline="2027-11-12 21:24:27.132950726 +0000 UTC" May 27 21:28:49.010773 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:49.010719 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12815h55m38.12223663s" May 27 21:28:49.109350 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:49.109283 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-h4fkz" event={"ID":"c33b5127-3c83-44e2-a1e0-5c0e433c9f00","Type":"ContainerStarted","Data":"0b03e93cafc2da8e2fcaf62cd725c0178be4ee8e78704cd6c37d5380a7268876"} May 27 21:28:49.115979 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:49.115948 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4k8dw" event={"ID":"74672ea1-1457-40d1-b8e1-5acca042ca3e","Type":"ContainerStarted","Data":"16e402be3182de1a41311bdd78e5a1497fcb61e9f33ebb0b8ea2b4724aa0d4ca"} May 27 21:28:49.127656 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:49.127626 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-246.ec2.internal" event={"ID":"7e5e8096305be124754f9f48db3e823f","Type":"ContainerStarted","Data":"68dcb85c876e789af4f9914d69527dd8a35c9ef4ee6abe697f0d89d527fd9451"} May 27 21:28:49.135763 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:49.135731 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-sc6tb" event={"ID":"d895055d-d1d4-44a6-9024-8b244e769d5e","Type":"ContainerStarted","Data":"0b5d94235a2a7bf0a3a83cf5d450e328c5c2ac7068a996e1a404e35c33572d03"} May 27 21:28:49.151899 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:49.151866 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-nls7n" event={"ID":"faad2ffc-1823-4376-9f3a-c208b39219d9","Type":"ContainerStarted","Data":"ae191c612bbe40cab269f86dd746f13acd208644f169f642c2f9eac27165466a"} May 27 21:28:49.170072 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:49.170042 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-knn5k" event={"ID":"bc955a1c-63ba-4921-8127-a0f27ae757e3","Type":"ContainerStarted","Data":"2c70519cdbffc16d2c568975e6a23557881d299bc0fc55f15492cdb3fd3d3146"} May 27 21:28:49.187576 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:49.187504 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mdd25" event={"ID":"e176311e-f128-4137-904b-7dd6fea5eed1","Type":"ContainerStarted","Data":"6cfa50d2c881d44e1100b2c4ba34ce5328b2445e1a7fb32d438ccb631b45db99"} May 27 21:28:49.194851 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:49.194818 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-246.ec2.internal" event={"ID":"6614026242b9bed0ac8a71fe40323c9f","Type":"ContainerStarted","Data":"9e5ecdf19dd2259bf4c9fc0bf3e97c8397cac55bbf3fb6b778bb2892d2259618"} May 27 21:28:49.208592 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:49.208558 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-sm52n" event={"ID":"e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a","Type":"ContainerStarted","Data":"626406db0126fc1fb3edbbd19373506e945f1657b5662f7ddc11d0cde9cbdb32"} May 27 21:28:49.233839 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:49.233712 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-p6vj8" event={"ID":"657348d2-3465-4003-9ef6-3139dc8c1a92","Type":"ContainerStarted","Data":"0c89cc93a568d0fb76b293fb04c347dc508346a75bbd9e018a6123df007ad7eb"} May 27 21:28:49.249956 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:49.249893 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" event={"ID":"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5","Type":"ContainerStarted","Data":"4cfeed8fc04c04d48de1ee3d45f0be53e393e41185ed43ccac9a1458a9fef329"} May 27 21:28:49.308642 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:49.308602 2564 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" May 27 21:28:49.584945 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:49.584833 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7fc93ab3-b601-4fe7-b073-b8702a09404d-metrics-certs\") pod \"network-metrics-daemon-gxt5q\" (UID: \"7fc93ab3-b601-4fe7-b073-b8702a09404d\") " pod="openshift-multus/network-metrics-daemon-gxt5q" May 27 21:28:49.585102 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:49.584985 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 27 21:28:49.585102 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:49.585047 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fc93ab3-b601-4fe7-b073-b8702a09404d-metrics-certs podName:7fc93ab3-b601-4fe7-b073-b8702a09404d nodeName:}" failed. No retries permitted until 2026-05-27 21:28:51.585029322 +0000 UTC m=+5.059234029 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7fc93ab3-b601-4fe7-b073-b8702a09404d-metrics-certs") pod "network-metrics-daemon-gxt5q" (UID: "7fc93ab3-b601-4fe7-b073-b8702a09404d") : object "openshift-multus"/"metrics-daemon-secret" not registered May 27 21:28:49.787668 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:49.787040 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p5cwl\" (UniqueName: \"kubernetes.io/projected/6dd8b333-153e-4494-a210-8d7ab405d008-kube-api-access-p5cwl\") pod \"network-check-target-2kzbj\" (UID: \"6dd8b333-153e-4494-a210-8d7ab405d008\") " pod="openshift-network-diagnostics/network-check-target-2kzbj" May 27 21:28:49.787668 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:49.787215 2564 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 27 21:28:49.787668 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:49.787236 2564 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 27 21:28:49.787668 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:49.787248 2564 projected.go:194] Error preparing data for projected volume kube-api-access-p5cwl for pod openshift-network-diagnostics/network-check-target-2kzbj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 27 21:28:49.787668 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:49.787321 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6dd8b333-153e-4494-a210-8d7ab405d008-kube-api-access-p5cwl podName:6dd8b333-153e-4494-a210-8d7ab405d008 nodeName:}" failed. No retries permitted until 2026-05-27 21:28:51.787302643 +0000 UTC m=+5.261507365 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-p5cwl" (UniqueName: "kubernetes.io/projected/6dd8b333-153e-4494-a210-8d7ab405d008-kube-api-access-p5cwl") pod "network-check-target-2kzbj" (UID: "6dd8b333-153e-4494-a210-8d7ab405d008") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 27 21:28:50.010959 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:50.010867 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-26 21:23:47 +0000 UTC" deadline="2028-03-06 23:10:12.508297076 +0000 UTC" May 27 21:28:50.010959 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:50.010911 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15577h41m22.497391422s" May 27 21:28:50.079845 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:50.079812 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2kzbj" May 27 21:28:50.080007 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:50.079949 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2kzbj" podUID="6dd8b333-153e-4494-a210-8d7ab405d008" May 27 21:28:50.080455 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:50.080435 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gxt5q" May 27 21:28:50.080591 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:50.080546 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gxt5q" podUID="7fc93ab3-b601-4fe7-b073-b8702a09404d" May 27 21:28:51.600496 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:51.600460 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7fc93ab3-b601-4fe7-b073-b8702a09404d-metrics-certs\") pod \"network-metrics-daemon-gxt5q\" (UID: \"7fc93ab3-b601-4fe7-b073-b8702a09404d\") " pod="openshift-multus/network-metrics-daemon-gxt5q" May 27 21:28:51.600927 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:51.600623 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 27 21:28:51.600927 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:51.600692 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fc93ab3-b601-4fe7-b073-b8702a09404d-metrics-certs podName:7fc93ab3-b601-4fe7-b073-b8702a09404d nodeName:}" failed. No retries permitted until 2026-05-27 21:28:55.600672921 +0000 UTC m=+9.074877629 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7fc93ab3-b601-4fe7-b073-b8702a09404d-metrics-certs") pod "network-metrics-daemon-gxt5q" (UID: "7fc93ab3-b601-4fe7-b073-b8702a09404d") : object "openshift-multus"/"metrics-daemon-secret" not registered May 27 21:28:51.802436 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:51.802398 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p5cwl\" (UniqueName: \"kubernetes.io/projected/6dd8b333-153e-4494-a210-8d7ab405d008-kube-api-access-p5cwl\") pod \"network-check-target-2kzbj\" (UID: \"6dd8b333-153e-4494-a210-8d7ab405d008\") " pod="openshift-network-diagnostics/network-check-target-2kzbj" May 27 21:28:51.802603 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:51.802569 2564 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 27 21:28:51.802603 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:51.802594 2564 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 27 21:28:51.802704 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:51.802606 2564 projected.go:194] Error preparing data for projected volume kube-api-access-p5cwl for pod openshift-network-diagnostics/network-check-target-2kzbj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 27 21:28:51.802704 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:51.802666 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6dd8b333-153e-4494-a210-8d7ab405d008-kube-api-access-p5cwl podName:6dd8b333-153e-4494-a210-8d7ab405d008 nodeName:}" failed. No retries permitted until 2026-05-27 21:28:55.80264813 +0000 UTC m=+9.276852841 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-p5cwl" (UniqueName: "kubernetes.io/projected/6dd8b333-153e-4494-a210-8d7ab405d008-kube-api-access-p5cwl") pod "network-check-target-2kzbj" (UID: "6dd8b333-153e-4494-a210-8d7ab405d008") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 27 21:28:52.080300 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:52.079906 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2kzbj" May 27 21:28:52.080300 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:52.080032 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2kzbj" podUID="6dd8b333-153e-4494-a210-8d7ab405d008" May 27 21:28:52.080300 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:52.080098 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gxt5q" May 27 21:28:52.080300 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:52.080219 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gxt5q" podUID="7fc93ab3-b601-4fe7-b073-b8702a09404d" May 27 21:28:54.079132 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:54.079023 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2kzbj" May 27 21:28:54.079132 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:54.079029 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gxt5q" May 27 21:28:54.079726 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:54.079164 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2kzbj" podUID="6dd8b333-153e-4494-a210-8d7ab405d008" May 27 21:28:54.079726 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:54.079253 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gxt5q" podUID="7fc93ab3-b601-4fe7-b073-b8702a09404d" May 27 21:28:55.635623 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:55.635574 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7fc93ab3-b601-4fe7-b073-b8702a09404d-metrics-certs\") pod \"network-metrics-daemon-gxt5q\" (UID: \"7fc93ab3-b601-4fe7-b073-b8702a09404d\") " pod="openshift-multus/network-metrics-daemon-gxt5q" May 27 21:28:55.636104 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:55.635745 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 27 21:28:55.636104 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:55.635811 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fc93ab3-b601-4fe7-b073-b8702a09404d-metrics-certs podName:7fc93ab3-b601-4fe7-b073-b8702a09404d nodeName:}" failed. No retries permitted until 2026-05-27 21:29:03.635791088 +0000 UTC m=+17.109995800 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7fc93ab3-b601-4fe7-b073-b8702a09404d-metrics-certs") pod "network-metrics-daemon-gxt5q" (UID: "7fc93ab3-b601-4fe7-b073-b8702a09404d") : object "openshift-multus"/"metrics-daemon-secret" not registered May 27 21:28:55.836830 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:55.836789 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p5cwl\" (UniqueName: \"kubernetes.io/projected/6dd8b333-153e-4494-a210-8d7ab405d008-kube-api-access-p5cwl\") pod \"network-check-target-2kzbj\" (UID: \"6dd8b333-153e-4494-a210-8d7ab405d008\") " pod="openshift-network-diagnostics/network-check-target-2kzbj" May 27 21:28:55.836999 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:55.836942 2564 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 27 21:28:55.836999 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:55.836959 2564 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 27 21:28:55.836999 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:55.836971 2564 projected.go:194] Error preparing data for projected volume kube-api-access-p5cwl for pod openshift-network-diagnostics/network-check-target-2kzbj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 27 21:28:55.837150 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:55.837027 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6dd8b333-153e-4494-a210-8d7ab405d008-kube-api-access-p5cwl podName:6dd8b333-153e-4494-a210-8d7ab405d008 nodeName:}" failed. No retries permitted until 2026-05-27 21:29:03.837010512 +0000 UTC m=+17.311215222 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-p5cwl" (UniqueName: "kubernetes.io/projected/6dd8b333-153e-4494-a210-8d7ab405d008-kube-api-access-p5cwl") pod "network-check-target-2kzbj" (UID: "6dd8b333-153e-4494-a210-8d7ab405d008") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 27 21:28:56.080163 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:56.079865 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2kzbj" May 27 21:28:56.080163 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:56.080005 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2kzbj" podUID="6dd8b333-153e-4494-a210-8d7ab405d008" May 27 21:28:56.080163 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:56.079867 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gxt5q" May 27 21:28:56.080463 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:56.080246 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gxt5q" podUID="7fc93ab3-b601-4fe7-b073-b8702a09404d" May 27 21:28:58.079948 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:58.079762 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gxt5q" May 27 21:28:58.080380 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:28:58.079774 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2kzbj" May 27 21:28:58.080380 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:58.080040 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gxt5q" podUID="7fc93ab3-b601-4fe7-b073-b8702a09404d" May 27 21:28:58.080380 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:28:58.080100 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2kzbj" podUID="6dd8b333-153e-4494-a210-8d7ab405d008" May 27 21:29:00.079813 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:00.079782 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2kzbj" May 27 21:29:00.080303 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:00.079788 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gxt5q" May 27 21:29:00.080303 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:00.079906 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2kzbj" podUID="6dd8b333-153e-4494-a210-8d7ab405d008" May 27 21:29:00.080303 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:00.079985 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gxt5q" podUID="7fc93ab3-b601-4fe7-b073-b8702a09404d" May 27 21:29:02.079968 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:02.079924 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2kzbj" May 27 21:29:02.080332 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:02.079934 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gxt5q" May 27 21:29:02.080332 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:02.080046 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2kzbj" podUID="6dd8b333-153e-4494-a210-8d7ab405d008" May 27 21:29:02.080332 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:02.080099 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gxt5q" podUID="7fc93ab3-b601-4fe7-b073-b8702a09404d" May 27 21:29:03.692018 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:03.691975 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7fc93ab3-b601-4fe7-b073-b8702a09404d-metrics-certs\") pod \"network-metrics-daemon-gxt5q\" (UID: \"7fc93ab3-b601-4fe7-b073-b8702a09404d\") " pod="openshift-multus/network-metrics-daemon-gxt5q" May 27 21:29:03.692568 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:03.692125 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 27 21:29:03.692568 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:03.692201 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fc93ab3-b601-4fe7-b073-b8702a09404d-metrics-certs podName:7fc93ab3-b601-4fe7-b073-b8702a09404d nodeName:}" failed. No retries permitted until 2026-05-27 21:29:19.692181526 +0000 UTC m=+33.166386247 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7fc93ab3-b601-4fe7-b073-b8702a09404d-metrics-certs") pod "network-metrics-daemon-gxt5q" (UID: "7fc93ab3-b601-4fe7-b073-b8702a09404d") : object "openshift-multus"/"metrics-daemon-secret" not registered May 27 21:29:03.893201 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:03.893159 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p5cwl\" (UniqueName: \"kubernetes.io/projected/6dd8b333-153e-4494-a210-8d7ab405d008-kube-api-access-p5cwl\") pod \"network-check-target-2kzbj\" (UID: \"6dd8b333-153e-4494-a210-8d7ab405d008\") " pod="openshift-network-diagnostics/network-check-target-2kzbj" May 27 21:29:03.893397 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:03.893325 2564 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 27 21:29:03.893397 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:03.893340 2564 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 27 21:29:03.893397 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:03.893350 2564 projected.go:194] Error preparing data for projected volume kube-api-access-p5cwl for pod openshift-network-diagnostics/network-check-target-2kzbj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 27 21:29:03.893518 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:03.893403 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6dd8b333-153e-4494-a210-8d7ab405d008-kube-api-access-p5cwl podName:6dd8b333-153e-4494-a210-8d7ab405d008 nodeName:}" failed. No retries permitted until 2026-05-27 21:29:19.893388228 +0000 UTC m=+33.367592935 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-p5cwl" (UniqueName: "kubernetes.io/projected/6dd8b333-153e-4494-a210-8d7ab405d008-kube-api-access-p5cwl") pod "network-check-target-2kzbj" (UID: "6dd8b333-153e-4494-a210-8d7ab405d008") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 27 21:29:04.079945 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:04.079855 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2kzbj" May 27 21:29:04.079945 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:04.079855 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gxt5q" May 27 21:29:04.080217 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:04.079991 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2kzbj" podUID="6dd8b333-153e-4494-a210-8d7ab405d008" May 27 21:29:04.080217 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:04.080139 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gxt5q" podUID="7fc93ab3-b601-4fe7-b073-b8702a09404d" May 27 21:29:06.079915 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:06.079874 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2kzbj" May 27 21:29:06.080383 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:06.079874 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gxt5q" May 27 21:29:06.080383 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:06.080010 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2kzbj" podUID="6dd8b333-153e-4494-a210-8d7ab405d008" May 27 21:29:06.080383 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:06.080092 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gxt5q" podUID="7fc93ab3-b601-4fe7-b073-b8702a09404d" May 27 21:29:07.296752 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:07.296518 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-nls7n" event={"ID":"faad2ffc-1823-4376-9f3a-c208b39219d9","Type":"ContainerStarted","Data":"db41af5f6bfb64bcf8f591f488ce375063b580982817a669cc0522d3dadb8cab"} May 27 21:29:07.298190 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:07.298166 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-246.ec2.internal" event={"ID":"6614026242b9bed0ac8a71fe40323c9f","Type":"ContainerStarted","Data":"b5f1d19b44c08d103b383945532d007ee4db327f95d0c729ddaffc9f703209c1"} May 27 21:29:07.299904 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:07.299866 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-sm52n" event={"ID":"e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a","Type":"ContainerStarted","Data":"ec01673bf6606f145685191ce1c717d921c3a27ca3508ab7975fc38d79a410e8"} May 27 21:29:07.302033 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:07.302011 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" event={"ID":"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5","Type":"ContainerStarted","Data":"55f80ae72ac209cb4c7b42af34e426b51ecf6c6fe20ba6c9b4634c6a9b9f5f51"} May 27 21:29:07.302093 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:07.302043 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" event={"ID":"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5","Type":"ContainerStarted","Data":"00dd409271c8eb7b6eabc3b772bf7f4acfe72a6fa2b0c5833f86378a875d5a0f"} May 27 21:29:07.302093 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:07.302053 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" event={"ID":"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5","Type":"ContainerStarted","Data":"be647d9fcb076d32deac646f9ea35452d64c47cc6cd08cb771ea5aafc217d62f"} May 27 21:29:07.331785 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:07.331438 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-246.ec2.internal" podStartSLOduration=20.331423325 podStartE2EDuration="20.331423325s" podCreationTimestamp="2026-05-27 21:28:47 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-27 21:29:07.331036084 +0000 UTC m=+20.805240814" watchObservedRunningTime="2026-05-27 21:29:07.331423325 +0000 UTC m=+20.805628053" May 27 21:29:07.331923 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:07.331788 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-nls7n" podStartSLOduration=1.9688472 podStartE2EDuration="20.331778351s" podCreationTimestamp="2026-05-27 21:28:47 +0000 UTC" firstStartedPulling="2026-05-27 21:28:48.41986135 +0000 UTC m=+1.894066057" lastFinishedPulling="2026-05-27 21:29:06.782792487 +0000 UTC m=+20.256997208" observedRunningTime="2026-05-27 21:29:07.316449806 +0000 UTC m=+20.790654535" watchObservedRunningTime="2026-05-27 21:29:07.331778351 +0000 UTC m=+20.805983086" May 27 21:29:07.347499 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:07.347463 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-sm52n" podStartSLOduration=1.974549472 podStartE2EDuration="20.347443329s" podCreationTimestamp="2026-05-27 21:28:47 +0000 UTC" firstStartedPulling="2026-05-27 21:28:48.419689278 +0000 UTC m=+1.893893985" lastFinishedPulling="2026-05-27 21:29:06.792583132 +0000 UTC m=+20.266787842" observedRunningTime="2026-05-27 21:29:07.347172073 +0000 UTC m=+20.821376796" watchObservedRunningTime="2026-05-27 21:29:07.347443329 +0000 UTC m=+20.821648057" May 27 21:29:08.079672 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:08.079633 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2kzbj" May 27 21:29:08.079833 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:08.079743 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2kzbj" podUID="6dd8b333-153e-4494-a210-8d7ab405d008" May 27 21:29:08.079833 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:08.079800 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gxt5q" May 27 21:29:08.079951 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:08.079920 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gxt5q" podUID="7fc93ab3-b601-4fe7-b073-b8702a09404d" May 27 21:29:08.304738 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:08.304700 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-p6vj8" event={"ID":"657348d2-3465-4003-9ef6-3139dc8c1a92","Type":"ContainerStarted","Data":"80b5c19631abf478b81915fba80a4d8be222b23f8da99f7f8c8218d2014bbb02"} May 27 21:29:08.307208 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:08.307185 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" event={"ID":"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5","Type":"ContainerStarted","Data":"c377cb68de4c9787eb461ceb1f632eaf7c1cb01bcc9fefe31dc9d9f7f630012c"} May 27 21:29:08.307337 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:08.307216 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" event={"ID":"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5","Type":"ContainerStarted","Data":"6fb76177d0b6f481af5cd319dcaf4627d1267f98239d9fa6104e5b6f39479317"} May 27 21:29:08.307337 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:08.307232 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" event={"ID":"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5","Type":"ContainerStarted","Data":"24d413f9baa989d562e16353fbac0d8298f467e7eefd42c59783730f92afa8de"} May 27 21:29:08.308451 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:08.308421 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-h4fkz" event={"ID":"c33b5127-3c83-44e2-a1e0-5c0e433c9f00","Type":"ContainerStarted","Data":"7a9ef0d429e5ba37dc991ec33c09f7675df6b40ff5260cccaedadc4ea8ab6467"} May 27 21:29:08.309802 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:08.309783 2564 generic.go:358] "Generic (PLEG): container finished" podID="74672ea1-1457-40d1-b8e1-5acca042ca3e" containerID="f40e0ecf0952c52bd642eaca48d7a1916a6f636b6adc39baffa4efd4adb4ccce" exitCode=0 May 27 21:29:08.309881 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:08.309841 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4k8dw" event={"ID":"74672ea1-1457-40d1-b8e1-5acca042ca3e","Type":"ContainerDied","Data":"f40e0ecf0952c52bd642eaca48d7a1916a6f636b6adc39baffa4efd4adb4ccce"} May 27 21:29:08.311202 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:08.311182 2564 generic.go:358] "Generic (PLEG): container finished" podID="7e5e8096305be124754f9f48db3e823f" containerID="2786f7df406c23c761adce116855faf6fdf76d385d868a3f633a91060a1b4e11" exitCode=0 May 27 21:29:08.311281 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:08.311228 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-246.ec2.internal" event={"ID":"7e5e8096305be124754f9f48db3e823f","Type":"ContainerDied","Data":"2786f7df406c23c761adce116855faf6fdf76d385d868a3f633a91060a1b4e11"} May 27 21:29:08.312588 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:08.312571 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-sc6tb" event={"ID":"d895055d-d1d4-44a6-9024-8b244e769d5e","Type":"ContainerStarted","Data":"7cc4678f3e2bb8843ec1c6171c8580abfec2f5c8d0fc3277c71c116e8f6b4b24"} May 27 21:29:08.313754 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:08.313735 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-knn5k" event={"ID":"bc955a1c-63ba-4921-8127-a0f27ae757e3","Type":"ContainerStarted","Data":"69667c7ff484c0e67c5489cae5805a7d81d5715a127707e9f6e496938a0afd45"} May 27 21:29:08.314939 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:08.314917 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mdd25" event={"ID":"e176311e-f128-4137-904b-7dd6fea5eed1","Type":"ContainerStarted","Data":"80cf6e7ac19cd21c5fee54cbaec7efb6264ba9c45ab8d701ec729c16c44d1fcf"} May 27 21:29:08.321361 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:08.321328 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-p6vj8" podStartSLOduration=2.897091231 podStartE2EDuration="21.321318693s" podCreationTimestamp="2026-05-27 21:28:47 +0000 UTC" firstStartedPulling="2026-05-27 21:28:48.355225019 +0000 UTC m=+1.829429726" lastFinishedPulling="2026-05-27 21:29:06.779452481 +0000 UTC m=+20.253657188" observedRunningTime="2026-05-27 21:29:08.320310103 +0000 UTC m=+21.794514832" watchObservedRunningTime="2026-05-27 21:29:08.321318693 +0000 UTC m=+21.795523422" May 27 21:29:08.348135 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:08.348090 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-mdd25" podStartSLOduration=2.901587553 podStartE2EDuration="21.348076861s" podCreationTimestamp="2026-05-27 21:28:47 +0000 UTC" firstStartedPulling="2026-05-27 21:28:48.290595227 +0000 UTC m=+1.764799934" lastFinishedPulling="2026-05-27 21:29:06.73708452 +0000 UTC m=+20.211289242" observedRunningTime="2026-05-27 21:29:08.33483102 +0000 UTC m=+21.809035748" watchObservedRunningTime="2026-05-27 21:29:08.348076861 +0000 UTC m=+21.822281591" May 27 21:29:08.348375 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:08.348348 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-h4fkz" podStartSLOduration=2.919979648 podStartE2EDuration="21.348340716s" podCreationTimestamp="2026-05-27 21:28:47 +0000 UTC" firstStartedPulling="2026-05-27 21:28:48.330560211 +0000 UTC m=+1.804764918" lastFinishedPulling="2026-05-27 21:29:06.758921269 +0000 UTC m=+20.233125986" observedRunningTime="2026-05-27 21:29:08.348178088 +0000 UTC m=+21.822382817" watchObservedRunningTime="2026-05-27 21:29:08.348340716 +0000 UTC m=+21.822545444" May 27 21:29:08.363691 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:08.363649 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-sc6tb" podStartSLOduration=3.043729024 podStartE2EDuration="21.363635742s" podCreationTimestamp="2026-05-27 21:28:47 +0000 UTC" firstStartedPulling="2026-05-27 21:28:48.439013662 +0000 UTC m=+1.913218372" lastFinishedPulling="2026-05-27 21:29:06.758920382 +0000 UTC m=+20.233125090" observedRunningTime="2026-05-27 21:29:08.3630366 +0000 UTC m=+21.837241340" watchObservedRunningTime="2026-05-27 21:29:08.363635742 +0000 UTC m=+21.837840471" May 27 21:29:08.746005 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:08.745982 2564 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" May 27 21:29:09.025727 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:09.025627 2564 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-05-27T21:29:08.745999253Z","UUID":"6eb77a98-48fd-4c29-ab90-b03d3a9b4bc8","Handler":null,"Name":"","Endpoint":""} May 27 21:29:09.027388 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:09.027365 2564 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 May 27 21:29:09.027388 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:09.027391 2564 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock May 27 21:29:09.318585 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:09.318505 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-knn5k" event={"ID":"bc955a1c-63ba-4921-8127-a0f27ae757e3","Type":"ContainerStarted","Data":"2f69da186e0b2486ca0ff229d8ad95a238c662942c54126ea05a2b4e086732e6"} May 27 21:29:09.319901 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:09.319865 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-246.ec2.internal" event={"ID":"7e5e8096305be124754f9f48db3e823f","Type":"ContainerStarted","Data":"c647d848b3992998697052ca2481c4e1b6a0e7f120f29cb5c090ef977d762d30"} May 27 21:29:09.336449 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:09.336410 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-246.ec2.internal" podStartSLOduration=22.336399662 podStartE2EDuration="22.336399662s" podCreationTimestamp="2026-05-27 21:28:47 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-27 21:29:09.336187245 +0000 UTC m=+22.810391972" watchObservedRunningTime="2026-05-27 21:29:09.336399662 +0000 UTC m=+22.810604390" May 27 21:29:10.080054 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:10.080018 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2kzbj" May 27 21:29:10.080229 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:10.080122 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2kzbj" podUID="6dd8b333-153e-4494-a210-8d7ab405d008" May 27 21:29:10.080229 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:10.080177 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gxt5q" May 27 21:29:10.080338 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:10.080261 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gxt5q" podUID="7fc93ab3-b601-4fe7-b073-b8702a09404d" May 27 21:29:10.325114 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:10.325079 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" event={"ID":"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5","Type":"ContainerStarted","Data":"fd1e321016d9e93fa49a6e0d952ebc6b741df5436a50d325d60344bc1274beda"} May 27 21:29:10.326900 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:10.326871 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-knn5k" event={"ID":"bc955a1c-63ba-4921-8127-a0f27ae757e3","Type":"ContainerStarted","Data":"95be77173b1651e2677be9810e597b091e8cb24b8cc4de52684f7d4b01a4167d"} May 27 21:29:10.347753 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:10.347713 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-knn5k" podStartSLOduration=1.567878004 podStartE2EDuration="23.347702655s" podCreationTimestamp="2026-05-27 21:28:47 +0000 UTC" firstStartedPulling="2026-05-27 21:28:48.361559464 +0000 UTC m=+1.835764171" lastFinishedPulling="2026-05-27 21:29:10.141384114 +0000 UTC m=+23.615588822" observedRunningTime="2026-05-27 21:29:10.347291066 +0000 UTC m=+23.821495797" watchObservedRunningTime="2026-05-27 21:29:10.347702655 +0000 UTC m=+23.821907386" May 27 21:29:10.500331 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:10.500299 2564 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-p6vj8" May 27 21:29:10.500893 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:10.500869 2564 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-p6vj8" May 27 21:29:11.329303 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:11.329258 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-p6vj8" May 27 21:29:11.329684 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:11.329627 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-p6vj8" May 27 21:29:12.079603 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:12.079392 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2kzbj" May 27 21:29:12.079808 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:12.079406 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gxt5q" May 27 21:29:12.079808 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:12.079712 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2kzbj" podUID="6dd8b333-153e-4494-a210-8d7ab405d008" May 27 21:29:12.079992 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:12.079801 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gxt5q" podUID="7fc93ab3-b601-4fe7-b073-b8702a09404d" May 27 21:29:12.334607 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:12.334510 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" event={"ID":"fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5","Type":"ContainerStarted","Data":"92eb6e5c3ae08dc45ba59386bcc171dcc1f6d923570acb103f203563e08bbe20"} May 27 21:29:12.335362 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:12.334830 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:29:12.335362 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:12.334862 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:29:12.335362 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:12.334875 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:29:12.350988 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:12.350878 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:29:12.351124 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:12.351110 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:29:12.363721 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:12.363679 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" podStartSLOduration=6.717781601 podStartE2EDuration="25.363666182s" podCreationTimestamp="2026-05-27 21:28:47 +0000 UTC" firstStartedPulling="2026-05-27 21:28:48.336936954 +0000 UTC m=+1.811141661" lastFinishedPulling="2026-05-27 21:29:06.982821534 +0000 UTC m=+20.457026242" observedRunningTime="2026-05-27 21:29:12.363040812 +0000 UTC m=+25.837245543" watchObservedRunningTime="2026-05-27 21:29:12.363666182 +0000 UTC m=+25.837870910" May 27 21:29:13.956787 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:13.956605 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-gxt5q"] May 27 21:29:13.957148 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:13.956895 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gxt5q" May 27 21:29:13.957148 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:13.957016 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gxt5q" podUID="7fc93ab3-b601-4fe7-b073-b8702a09404d" May 27 21:29:13.959256 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:13.959224 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2kzbj"] May 27 21:29:13.959393 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:13.959338 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2kzbj" May 27 21:29:13.959447 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:13.959432 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2kzbj" podUID="6dd8b333-153e-4494-a210-8d7ab405d008" May 27 21:29:15.079453 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:15.079418 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2kzbj" May 27 21:29:15.079941 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:15.079553 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2kzbj" podUID="6dd8b333-153e-4494-a210-8d7ab405d008" May 27 21:29:15.340866 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:15.340777 2564 generic.go:358] "Generic (PLEG): container finished" podID="74672ea1-1457-40d1-b8e1-5acca042ca3e" containerID="038a73861720090ca960a110ae662092043634e4385897d269522c0c4ff93fe6" exitCode=0 May 27 21:29:15.340866 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:15.340819 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4k8dw" event={"ID":"74672ea1-1457-40d1-b8e1-5acca042ca3e","Type":"ContainerDied","Data":"038a73861720090ca960a110ae662092043634e4385897d269522c0c4ff93fe6"} May 27 21:29:16.079861 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:16.079831 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gxt5q" May 27 21:29:16.080258 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:16.079962 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gxt5q" podUID="7fc93ab3-b601-4fe7-b073-b8702a09404d" May 27 21:29:17.079799 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:17.079647 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2kzbj" May 27 21:29:17.079984 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:17.079849 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2kzbj" podUID="6dd8b333-153e-4494-a210-8d7ab405d008" May 27 21:29:17.346292 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:17.346190 2564 generic.go:358] "Generic (PLEG): container finished" podID="74672ea1-1457-40d1-b8e1-5acca042ca3e" containerID="cdd4335014ecd6df186a0b0376ee4a9bc5a1a821a30072cf64cf880e01572b41" exitCode=0 May 27 21:29:17.346292 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:17.346225 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4k8dw" event={"ID":"74672ea1-1457-40d1-b8e1-5acca042ca3e","Type":"ContainerDied","Data":"cdd4335014ecd6df186a0b0376ee4a9bc5a1a821a30072cf64cf880e01572b41"} May 27 21:29:18.079959 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:18.079929 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gxt5q" May 27 21:29:18.080146 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:18.080034 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gxt5q" podUID="7fc93ab3-b601-4fe7-b073-b8702a09404d" May 27 21:29:19.079391 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:19.079364 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2kzbj" May 27 21:29:19.079553 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:19.079464 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2kzbj" podUID="6dd8b333-153e-4494-a210-8d7ab405d008" May 27 21:29:19.351811 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:19.351741 2564 generic.go:358] "Generic (PLEG): container finished" podID="74672ea1-1457-40d1-b8e1-5acca042ca3e" containerID="f1a912e90613a2a2fdb4c1c89419ce95a11e4492a190ca844dbb593c5f5f9241" exitCode=0 May 27 21:29:19.351811 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:19.351790 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4k8dw" event={"ID":"74672ea1-1457-40d1-b8e1-5acca042ca3e","Type":"ContainerDied","Data":"f1a912e90613a2a2fdb4c1c89419ce95a11e4492a190ca844dbb593c5f5f9241"} May 27 21:29:19.717538 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:19.717506 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7fc93ab3-b601-4fe7-b073-b8702a09404d-metrics-certs\") pod \"network-metrics-daemon-gxt5q\" (UID: \"7fc93ab3-b601-4fe7-b073-b8702a09404d\") " pod="openshift-multus/network-metrics-daemon-gxt5q" May 27 21:29:19.717679 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:19.717618 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 27 21:29:19.717679 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:19.717662 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fc93ab3-b601-4fe7-b073-b8702a09404d-metrics-certs podName:7fc93ab3-b601-4fe7-b073-b8702a09404d nodeName:}" failed. No retries permitted until 2026-05-27 21:29:51.717649859 +0000 UTC m=+65.191854566 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7fc93ab3-b601-4fe7-b073-b8702a09404d-metrics-certs") pod "network-metrics-daemon-gxt5q" (UID: "7fc93ab3-b601-4fe7-b073-b8702a09404d") : object "openshift-multus"/"metrics-daemon-secret" not registered May 27 21:29:19.886395 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:19.886369 2564 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-246.ec2.internal" event="NodeReady" May 27 21:29:19.886523 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:19.886455 2564 kubelet_node_status.go:550] "Fast updating node status as it just became ready" May 27 21:29:19.919704 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:19.919676 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p5cwl\" (UniqueName: \"kubernetes.io/projected/6dd8b333-153e-4494-a210-8d7ab405d008-kube-api-access-p5cwl\") pod \"network-check-target-2kzbj\" (UID: \"6dd8b333-153e-4494-a210-8d7ab405d008\") " pod="openshift-network-diagnostics/network-check-target-2kzbj" May 27 21:29:19.919869 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:19.919796 2564 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 27 21:29:19.919869 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:19.919812 2564 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 27 21:29:19.919869 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:19.919821 2564 projected.go:194] Error preparing data for projected volume kube-api-access-p5cwl for pod openshift-network-diagnostics/network-check-target-2kzbj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 27 21:29:19.920032 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:19.919872 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6dd8b333-153e-4494-a210-8d7ab405d008-kube-api-access-p5cwl podName:6dd8b333-153e-4494-a210-8d7ab405d008 nodeName:}" failed. No retries permitted until 2026-05-27 21:29:51.919857565 +0000 UTC m=+65.394062273 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-p5cwl" (UniqueName: "kubernetes.io/projected/6dd8b333-153e-4494-a210-8d7ab405d008-kube-api-access-p5cwl") pod "network-check-target-2kzbj" (UID: "6dd8b333-153e-4494-a210-8d7ab405d008") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 27 21:29:19.933486 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:19.933458 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-4t29f"] May 27 21:29:19.948948 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:19.948482 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-5ntj8"] May 27 21:29:19.949192 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:19.949169 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4t29f" May 27 21:29:19.951969 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:19.951940 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" May 27 21:29:19.952081 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:19.952014 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" May 27 21:29:19.952278 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:19.952252 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xztv5\"" May 27 21:29:19.963414 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:19.963396 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4t29f"] May 27 21:29:19.963506 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:19.963421 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5ntj8"] May 27 21:29:19.963555 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:19.963523 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5ntj8" May 27 21:29:19.966282 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:19.966243 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" May 27 21:29:19.966384 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:19.966309 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" May 27 21:29:19.966801 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:19.966781 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" May 27 21:29:19.967085 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:19.967065 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-vmd24\"" May 27 21:29:20.020968 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:20.020944 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxh4q\" (UniqueName: \"kubernetes.io/projected/f0c8b7dd-e25c-44d5-a1ac-52e4cef19011-kube-api-access-xxh4q\") pod \"dns-default-4t29f\" (UID: \"f0c8b7dd-e25c-44d5-a1ac-52e4cef19011\") " pod="openshift-dns/dns-default-4t29f" May 27 21:29:20.021103 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:20.020995 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f0c8b7dd-e25c-44d5-a1ac-52e4cef19011-metrics-tls\") pod \"dns-default-4t29f\" (UID: \"f0c8b7dd-e25c-44d5-a1ac-52e4cef19011\") " pod="openshift-dns/dns-default-4t29f" May 27 21:29:20.021103 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:20.021014 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f0c8b7dd-e25c-44d5-a1ac-52e4cef19011-config-volume\") pod \"dns-default-4t29f\" (UID: \"f0c8b7dd-e25c-44d5-a1ac-52e4cef19011\") " pod="openshift-dns/dns-default-4t29f" May 27 21:29:20.021103 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:20.021032 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0c8b7dd-e25c-44d5-a1ac-52e4cef19011-tmp-dir\") pod \"dns-default-4t29f\" (UID: \"f0c8b7dd-e25c-44d5-a1ac-52e4cef19011\") " pod="openshift-dns/dns-default-4t29f" May 27 21:29:20.080010 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:20.079982 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gxt5q" May 27 21:29:20.083064 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:20.083039 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-qdzf7\"" May 27 21:29:20.083198 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:20.083050 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" May 27 21:29:20.122097 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:20.122074 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0a23d452-20fc-4665-9d3a-ef17d210a8d9-cert\") pod \"ingress-canary-5ntj8\" (UID: \"0a23d452-20fc-4665-9d3a-ef17d210a8d9\") " pod="openshift-ingress-canary/ingress-canary-5ntj8" May 27 21:29:20.122204 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:20.122115 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xxh4q\" (UniqueName: \"kubernetes.io/projected/f0c8b7dd-e25c-44d5-a1ac-52e4cef19011-kube-api-access-xxh4q\") pod \"dns-default-4t29f\" (UID: \"f0c8b7dd-e25c-44d5-a1ac-52e4cef19011\") " pod="openshift-dns/dns-default-4t29f" May 27 21:29:20.122253 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:20.122195 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f0c8b7dd-e25c-44d5-a1ac-52e4cef19011-metrics-tls\") pod \"dns-default-4t29f\" (UID: \"f0c8b7dd-e25c-44d5-a1ac-52e4cef19011\") " pod="openshift-dns/dns-default-4t29f" May 27 21:29:20.122253 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:20.122239 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btf2j\" (UniqueName: \"kubernetes.io/projected/0a23d452-20fc-4665-9d3a-ef17d210a8d9-kube-api-access-btf2j\") pod \"ingress-canary-5ntj8\" (UID: \"0a23d452-20fc-4665-9d3a-ef17d210a8d9\") " pod="openshift-ingress-canary/ingress-canary-5ntj8" May 27 21:29:20.122373 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:20.122287 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f0c8b7dd-e25c-44d5-a1ac-52e4cef19011-config-volume\") pod \"dns-default-4t29f\" (UID: \"f0c8b7dd-e25c-44d5-a1ac-52e4cef19011\") " pod="openshift-dns/dns-default-4t29f" May 27 21:29:20.122373 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:20.122318 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0c8b7dd-e25c-44d5-a1ac-52e4cef19011-tmp-dir\") pod \"dns-default-4t29f\" (UID: \"f0c8b7dd-e25c-44d5-a1ac-52e4cef19011\") " pod="openshift-dns/dns-default-4t29f" May 27 21:29:20.122373 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:20.122351 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 27 21:29:20.122509 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:20.122407 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0c8b7dd-e25c-44d5-a1ac-52e4cef19011-metrics-tls podName:f0c8b7dd-e25c-44d5-a1ac-52e4cef19011 nodeName:}" failed. No retries permitted until 2026-05-27 21:29:20.622388853 +0000 UTC m=+34.096593569 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f0c8b7dd-e25c-44d5-a1ac-52e4cef19011-metrics-tls") pod "dns-default-4t29f" (UID: "f0c8b7dd-e25c-44d5-a1ac-52e4cef19011") : secret "dns-default-metrics-tls" not found May 27 21:29:20.122693 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:20.122675 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0c8b7dd-e25c-44d5-a1ac-52e4cef19011-tmp-dir\") pod \"dns-default-4t29f\" (UID: \"f0c8b7dd-e25c-44d5-a1ac-52e4cef19011\") " pod="openshift-dns/dns-default-4t29f" May 27 21:29:20.122833 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:20.122807 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f0c8b7dd-e25c-44d5-a1ac-52e4cef19011-config-volume\") pod \"dns-default-4t29f\" (UID: \"f0c8b7dd-e25c-44d5-a1ac-52e4cef19011\") " pod="openshift-dns/dns-default-4t29f" May 27 21:29:20.133345 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:20.133325 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxh4q\" (UniqueName: \"kubernetes.io/projected/f0c8b7dd-e25c-44d5-a1ac-52e4cef19011-kube-api-access-xxh4q\") pod \"dns-default-4t29f\" (UID: \"f0c8b7dd-e25c-44d5-a1ac-52e4cef19011\") " pod="openshift-dns/dns-default-4t29f" May 27 21:29:20.223463 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:20.223378 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-btf2j\" (UniqueName: \"kubernetes.io/projected/0a23d452-20fc-4665-9d3a-ef17d210a8d9-kube-api-access-btf2j\") pod \"ingress-canary-5ntj8\" (UID: \"0a23d452-20fc-4665-9d3a-ef17d210a8d9\") " pod="openshift-ingress-canary/ingress-canary-5ntj8" May 27 21:29:20.223463 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:20.223426 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0a23d452-20fc-4665-9d3a-ef17d210a8d9-cert\") pod \"ingress-canary-5ntj8\" (UID: \"0a23d452-20fc-4665-9d3a-ef17d210a8d9\") " pod="openshift-ingress-canary/ingress-canary-5ntj8" May 27 21:29:20.223636 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:20.223531 2564 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 27 21:29:20.223636 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:20.223592 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a23d452-20fc-4665-9d3a-ef17d210a8d9-cert podName:0a23d452-20fc-4665-9d3a-ef17d210a8d9 nodeName:}" failed. No retries permitted until 2026-05-27 21:29:20.723574507 +0000 UTC m=+34.197779219 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/0a23d452-20fc-4665-9d3a-ef17d210a8d9-cert") pod "ingress-canary-5ntj8" (UID: "0a23d452-20fc-4665-9d3a-ef17d210a8d9") : secret "canary-serving-cert" not found May 27 21:29:20.232731 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:20.232704 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-btf2j\" (UniqueName: \"kubernetes.io/projected/0a23d452-20fc-4665-9d3a-ef17d210a8d9-kube-api-access-btf2j\") pod \"ingress-canary-5ntj8\" (UID: \"0a23d452-20fc-4665-9d3a-ef17d210a8d9\") " pod="openshift-ingress-canary/ingress-canary-5ntj8" May 27 21:29:20.627126 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:20.627049 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f0c8b7dd-e25c-44d5-a1ac-52e4cef19011-metrics-tls\") pod \"dns-default-4t29f\" (UID: \"f0c8b7dd-e25c-44d5-a1ac-52e4cef19011\") " pod="openshift-dns/dns-default-4t29f" May 27 21:29:20.627622 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:20.627194 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 27 21:29:20.627622 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:20.627257 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0c8b7dd-e25c-44d5-a1ac-52e4cef19011-metrics-tls podName:f0c8b7dd-e25c-44d5-a1ac-52e4cef19011 nodeName:}" failed. No retries permitted until 2026-05-27 21:29:21.627239513 +0000 UTC m=+35.101444221 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f0c8b7dd-e25c-44d5-a1ac-52e4cef19011-metrics-tls") pod "dns-default-4t29f" (UID: "f0c8b7dd-e25c-44d5-a1ac-52e4cef19011") : secret "dns-default-metrics-tls" not found May 27 21:29:20.727549 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:20.727513 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0a23d452-20fc-4665-9d3a-ef17d210a8d9-cert\") pod \"ingress-canary-5ntj8\" (UID: \"0a23d452-20fc-4665-9d3a-ef17d210a8d9\") " pod="openshift-ingress-canary/ingress-canary-5ntj8" May 27 21:29:20.727714 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:20.727652 2564 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 27 21:29:20.727774 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:20.727721 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a23d452-20fc-4665-9d3a-ef17d210a8d9-cert podName:0a23d452-20fc-4665-9d3a-ef17d210a8d9 nodeName:}" failed. No retries permitted until 2026-05-27 21:29:21.727699381 +0000 UTC m=+35.201904095 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/0a23d452-20fc-4665-9d3a-ef17d210a8d9-cert") pod "ingress-canary-5ntj8" (UID: "0a23d452-20fc-4665-9d3a-ef17d210a8d9") : secret "canary-serving-cert" not found May 27 21:29:21.080021 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:21.079989 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2kzbj" May 27 21:29:21.083110 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:21.083088 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" May 27 21:29:21.083110 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:21.083101 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" May 27 21:29:21.083300 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:21.083154 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-s5jvq\"" May 27 21:29:21.634128 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:21.634094 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f0c8b7dd-e25c-44d5-a1ac-52e4cef19011-metrics-tls\") pod \"dns-default-4t29f\" (UID: \"f0c8b7dd-e25c-44d5-a1ac-52e4cef19011\") " pod="openshift-dns/dns-default-4t29f" May 27 21:29:21.634665 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:21.634238 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 27 21:29:21.634665 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:21.634322 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0c8b7dd-e25c-44d5-a1ac-52e4cef19011-metrics-tls podName:f0c8b7dd-e25c-44d5-a1ac-52e4cef19011 nodeName:}" failed. No retries permitted until 2026-05-27 21:29:23.634307125 +0000 UTC m=+37.108511836 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f0c8b7dd-e25c-44d5-a1ac-52e4cef19011-metrics-tls") pod "dns-default-4t29f" (UID: "f0c8b7dd-e25c-44d5-a1ac-52e4cef19011") : secret "dns-default-metrics-tls" not found May 27 21:29:21.735168 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:21.735135 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0a23d452-20fc-4665-9d3a-ef17d210a8d9-cert\") pod \"ingress-canary-5ntj8\" (UID: \"0a23d452-20fc-4665-9d3a-ef17d210a8d9\") " pod="openshift-ingress-canary/ingress-canary-5ntj8" May 27 21:29:21.735363 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:21.735302 2564 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 27 21:29:21.735426 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:21.735381 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a23d452-20fc-4665-9d3a-ef17d210a8d9-cert podName:0a23d452-20fc-4665-9d3a-ef17d210a8d9 nodeName:}" failed. No retries permitted until 2026-05-27 21:29:23.735364021 +0000 UTC m=+37.209568741 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/0a23d452-20fc-4665-9d3a-ef17d210a8d9-cert") pod "ingress-canary-5ntj8" (UID: "0a23d452-20fc-4665-9d3a-ef17d210a8d9") : secret "canary-serving-cert" not found May 27 21:29:23.651022 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:23.650986 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f0c8b7dd-e25c-44d5-a1ac-52e4cef19011-metrics-tls\") pod \"dns-default-4t29f\" (UID: \"f0c8b7dd-e25c-44d5-a1ac-52e4cef19011\") " pod="openshift-dns/dns-default-4t29f" May 27 21:29:23.651417 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:23.651168 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 27 21:29:23.651417 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:23.651244 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0c8b7dd-e25c-44d5-a1ac-52e4cef19011-metrics-tls podName:f0c8b7dd-e25c-44d5-a1ac-52e4cef19011 nodeName:}" failed. No retries permitted until 2026-05-27 21:29:27.651224088 +0000 UTC m=+41.125428801 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f0c8b7dd-e25c-44d5-a1ac-52e4cef19011-metrics-tls") pod "dns-default-4t29f" (UID: "f0c8b7dd-e25c-44d5-a1ac-52e4cef19011") : secret "dns-default-metrics-tls" not found May 27 21:29:23.752044 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:23.751967 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0a23d452-20fc-4665-9d3a-ef17d210a8d9-cert\") pod \"ingress-canary-5ntj8\" (UID: \"0a23d452-20fc-4665-9d3a-ef17d210a8d9\") " pod="openshift-ingress-canary/ingress-canary-5ntj8" May 27 21:29:23.752183 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:23.752086 2564 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 27 21:29:23.752183 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:23.752158 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a23d452-20fc-4665-9d3a-ef17d210a8d9-cert podName:0a23d452-20fc-4665-9d3a-ef17d210a8d9 nodeName:}" failed. No retries permitted until 2026-05-27 21:29:27.752140588 +0000 UTC m=+41.226345295 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/0a23d452-20fc-4665-9d3a-ef17d210a8d9-cert") pod "ingress-canary-5ntj8" (UID: "0a23d452-20fc-4665-9d3a-ef17d210a8d9") : secret "canary-serving-cert" not found May 27 21:29:26.368808 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:26.368778 2564 generic.go:358] "Generic (PLEG): container finished" podID="74672ea1-1457-40d1-b8e1-5acca042ca3e" containerID="66cf1259618a027b8be272d46695371f4f220b89abad581175c14c2c9b30eefa" exitCode=0 May 27 21:29:26.369124 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:26.368824 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4k8dw" event={"ID":"74672ea1-1457-40d1-b8e1-5acca042ca3e","Type":"ContainerDied","Data":"66cf1259618a027b8be272d46695371f4f220b89abad581175c14c2c9b30eefa"} May 27 21:29:27.373700 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:27.373667 2564 generic.go:358] "Generic (PLEG): container finished" podID="74672ea1-1457-40d1-b8e1-5acca042ca3e" containerID="ebfacf89449a97d1a32ed250d1631b2c3fd4ac4d5fdd1593e282118fc7fe3ecb" exitCode=0 May 27 21:29:27.374052 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:27.373716 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4k8dw" event={"ID":"74672ea1-1457-40d1-b8e1-5acca042ca3e","Type":"ContainerDied","Data":"ebfacf89449a97d1a32ed250d1631b2c3fd4ac4d5fdd1593e282118fc7fe3ecb"} May 27 21:29:27.677552 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:27.677524 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f0c8b7dd-e25c-44d5-a1ac-52e4cef19011-metrics-tls\") pod \"dns-default-4t29f\" (UID: \"f0c8b7dd-e25c-44d5-a1ac-52e4cef19011\") " pod="openshift-dns/dns-default-4t29f" May 27 21:29:27.677696 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:27.677666 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 27 21:29:27.677750 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:27.677732 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0c8b7dd-e25c-44d5-a1ac-52e4cef19011-metrics-tls podName:f0c8b7dd-e25c-44d5-a1ac-52e4cef19011 nodeName:}" failed. No retries permitted until 2026-05-27 21:29:35.677716002 +0000 UTC m=+49.151920731 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f0c8b7dd-e25c-44d5-a1ac-52e4cef19011-metrics-tls") pod "dns-default-4t29f" (UID: "f0c8b7dd-e25c-44d5-a1ac-52e4cef19011") : secret "dns-default-metrics-tls" not found May 27 21:29:27.778562 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:27.778536 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0a23d452-20fc-4665-9d3a-ef17d210a8d9-cert\") pod \"ingress-canary-5ntj8\" (UID: \"0a23d452-20fc-4665-9d3a-ef17d210a8d9\") " pod="openshift-ingress-canary/ingress-canary-5ntj8" May 27 21:29:27.778713 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:27.778639 2564 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 27 21:29:27.778713 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:27.778684 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a23d452-20fc-4665-9d3a-ef17d210a8d9-cert podName:0a23d452-20fc-4665-9d3a-ef17d210a8d9 nodeName:}" failed. No retries permitted until 2026-05-27 21:29:35.778671562 +0000 UTC m=+49.252876269 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/0a23d452-20fc-4665-9d3a-ef17d210a8d9-cert") pod "ingress-canary-5ntj8" (UID: "0a23d452-20fc-4665-9d3a-ef17d210a8d9") : secret "canary-serving-cert" not found May 27 21:29:28.380562 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:28.380526 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4k8dw" event={"ID":"74672ea1-1457-40d1-b8e1-5acca042ca3e","Type":"ContainerStarted","Data":"b08f285dea15b16a6cbd48feb45bad9988a1ccfce6281568a5158df7af0f3c55"} May 27 21:29:28.406617 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:28.406572 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-4k8dw" podStartSLOduration=3.6229772369999997 podStartE2EDuration="41.406556988s" podCreationTimestamp="2026-05-27 21:28:47 +0000 UTC" firstStartedPulling="2026-05-27 21:28:48.30382656 +0000 UTC m=+1.778031267" lastFinishedPulling="2026-05-27 21:29:26.087406306 +0000 UTC m=+39.561611018" observedRunningTime="2026-05-27 21:29:28.405121752 +0000 UTC m=+41.879326491" watchObservedRunningTime="2026-05-27 21:29:28.406556988 +0000 UTC m=+41.880761717" May 27 21:29:35.734338 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:35.734294 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f0c8b7dd-e25c-44d5-a1ac-52e4cef19011-metrics-tls\") pod \"dns-default-4t29f\" (UID: \"f0c8b7dd-e25c-44d5-a1ac-52e4cef19011\") " pod="openshift-dns/dns-default-4t29f" May 27 21:29:35.734708 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:35.734432 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 27 21:29:35.734708 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:35.734502 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0c8b7dd-e25c-44d5-a1ac-52e4cef19011-metrics-tls podName:f0c8b7dd-e25c-44d5-a1ac-52e4cef19011 nodeName:}" failed. No retries permitted until 2026-05-27 21:29:51.734482992 +0000 UTC m=+65.208687703 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f0c8b7dd-e25c-44d5-a1ac-52e4cef19011-metrics-tls") pod "dns-default-4t29f" (UID: "f0c8b7dd-e25c-44d5-a1ac-52e4cef19011") : secret "dns-default-metrics-tls" not found May 27 21:29:35.834802 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:35.834756 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0a23d452-20fc-4665-9d3a-ef17d210a8d9-cert\") pod \"ingress-canary-5ntj8\" (UID: \"0a23d452-20fc-4665-9d3a-ef17d210a8d9\") " pod="openshift-ingress-canary/ingress-canary-5ntj8" May 27 21:29:35.834932 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:35.834900 2564 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 27 21:29:35.834989 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:35.834968 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a23d452-20fc-4665-9d3a-ef17d210a8d9-cert podName:0a23d452-20fc-4665-9d3a-ef17d210a8d9 nodeName:}" failed. No retries permitted until 2026-05-27 21:29:51.834953531 +0000 UTC m=+65.309158385 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/0a23d452-20fc-4665-9d3a-ef17d210a8d9-cert") pod "ingress-canary-5ntj8" (UID: "0a23d452-20fc-4665-9d3a-ef17d210a8d9") : secret "canary-serving-cert" not found May 27 21:29:44.347480 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:44.347440 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8p7kk" May 27 21:29:51.746881 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:51.746825 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f0c8b7dd-e25c-44d5-a1ac-52e4cef19011-metrics-tls\") pod \"dns-default-4t29f\" (UID: \"f0c8b7dd-e25c-44d5-a1ac-52e4cef19011\") " pod="openshift-dns/dns-default-4t29f" May 27 21:29:51.746881 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:51.746886 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7fc93ab3-b601-4fe7-b073-b8702a09404d-metrics-certs\") pod \"network-metrics-daemon-gxt5q\" (UID: \"7fc93ab3-b601-4fe7-b073-b8702a09404d\") " pod="openshift-multus/network-metrics-daemon-gxt5q" May 27 21:29:51.747500 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:51.746978 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 27 21:29:51.747500 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:51.747045 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0c8b7dd-e25c-44d5-a1ac-52e4cef19011-metrics-tls podName:f0c8b7dd-e25c-44d5-a1ac-52e4cef19011 nodeName:}" failed. No retries permitted until 2026-05-27 21:30:23.747030048 +0000 UTC m=+97.221234755 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f0c8b7dd-e25c-44d5-a1ac-52e4cef19011-metrics-tls") pod "dns-default-4t29f" (UID: "f0c8b7dd-e25c-44d5-a1ac-52e4cef19011") : secret "dns-default-metrics-tls" not found May 27 21:29:51.750026 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:51.750007 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" May 27 21:29:51.757772 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:51.757754 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found May 27 21:29:51.757868 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:51.757807 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fc93ab3-b601-4fe7-b073-b8702a09404d-metrics-certs podName:7fc93ab3-b601-4fe7-b073-b8702a09404d nodeName:}" failed. No retries permitted until 2026-05-27 21:30:55.757789879 +0000 UTC m=+129.231994586 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7fc93ab3-b601-4fe7-b073-b8702a09404d-metrics-certs") pod "network-metrics-daemon-gxt5q" (UID: "7fc93ab3-b601-4fe7-b073-b8702a09404d") : secret "metrics-daemon-secret" not found May 27 21:29:51.847956 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:51.847905 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0a23d452-20fc-4665-9d3a-ef17d210a8d9-cert\") pod \"ingress-canary-5ntj8\" (UID: \"0a23d452-20fc-4665-9d3a-ef17d210a8d9\") " pod="openshift-ingress-canary/ingress-canary-5ntj8" May 27 21:29:51.848113 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:51.848005 2564 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 27 21:29:51.848113 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:29:51.848072 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a23d452-20fc-4665-9d3a-ef17d210a8d9-cert podName:0a23d452-20fc-4665-9d3a-ef17d210a8d9 nodeName:}" failed. No retries permitted until 2026-05-27 21:30:23.848054488 +0000 UTC m=+97.322259199 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/0a23d452-20fc-4665-9d3a-ef17d210a8d9-cert") pod "ingress-canary-5ntj8" (UID: "0a23d452-20fc-4665-9d3a-ef17d210a8d9") : secret "canary-serving-cert" not found May 27 21:29:51.948496 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:51.948464 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p5cwl\" (UniqueName: \"kubernetes.io/projected/6dd8b333-153e-4494-a210-8d7ab405d008-kube-api-access-p5cwl\") pod \"network-check-target-2kzbj\" (UID: \"6dd8b333-153e-4494-a210-8d7ab405d008\") " pod="openshift-network-diagnostics/network-check-target-2kzbj" May 27 21:29:51.954484 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:51.954467 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" May 27 21:29:51.962896 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:51.962882 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" May 27 21:29:51.972315 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:51.972289 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5cwl\" (UniqueName: \"kubernetes.io/projected/6dd8b333-153e-4494-a210-8d7ab405d008-kube-api-access-p5cwl\") pod \"network-check-target-2kzbj\" (UID: \"6dd8b333-153e-4494-a210-8d7ab405d008\") " pod="openshift-network-diagnostics/network-check-target-2kzbj" May 27 21:29:51.991460 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:51.991430 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-s5jvq\"" May 27 21:29:51.999450 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:51.999408 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2kzbj" May 27 21:29:52.127404 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:52.127382 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2kzbj"] May 27 21:29:52.129355 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:29:52.129321 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dd8b333_153e_4494_a210_8d7ab405d008.slice/crio-b18d670e5c4c21eaa9c4f27d239bdc2278192e8c7eccb3db5a4460e868c7bd90 WatchSource:0}: Error finding container b18d670e5c4c21eaa9c4f27d239bdc2278192e8c7eccb3db5a4460e868c7bd90: Status 404 returned error can't find the container with id b18d670e5c4c21eaa9c4f27d239bdc2278192e8c7eccb3db5a4460e868c7bd90 May 27 21:29:52.421883 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:52.421854 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2kzbj" event={"ID":"6dd8b333-153e-4494-a210-8d7ab405d008","Type":"ContainerStarted","Data":"b18d670e5c4c21eaa9c4f27d239bdc2278192e8c7eccb3db5a4460e868c7bd90"} May 27 21:29:55.428002 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:55.427968 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2kzbj" event={"ID":"6dd8b333-153e-4494-a210-8d7ab405d008","Type":"ContainerStarted","Data":"ce2f9016d901ecaf401bb3f7eb6448f068727ec49bd83c3b89211c18f18c2ba2"} May 27 21:29:55.428468 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:55.428094 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-2kzbj" May 27 21:29:55.445421 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:29:55.445381 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-2kzbj" podStartSLOduration=65.72092335 podStartE2EDuration="1m8.445368329s" podCreationTimestamp="2026-05-27 21:28:47 +0000 UTC" firstStartedPulling="2026-05-27 21:29:52.131142877 +0000 UTC m=+65.605347587" lastFinishedPulling="2026-05-27 21:29:54.85558784 +0000 UTC m=+68.329792566" observedRunningTime="2026-05-27 21:29:55.444771553 +0000 UTC m=+68.918976282" watchObservedRunningTime="2026-05-27 21:29:55.445368329 +0000 UTC m=+68.919573082" May 27 21:30:20.970714 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:20.970598 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-c5dbz"] May 27 21:30:20.973401 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:20.973376 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-c5dbz" May 27 21:30:20.976092 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:20.976063 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" May 27 21:30:20.976207 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:20.976105 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" May 27 21:30:20.977177 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:20.977160 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-54lw5\"" May 27 21:30:20.989758 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:20.989735 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-c5dbz"] May 27 21:30:21.033771 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.031466 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rjxw\" (UniqueName: \"kubernetes.io/projected/3bdeafef-3082-4f0e-8add-fcdf60551c63-kube-api-access-4rjxw\") pod \"volume-data-source-validator-5cf8c89d64-c5dbz\" (UID: \"3bdeafef-3082-4f0e-8add-fcdf60551c63\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-c5dbz" May 27 21:30:21.098406 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.098373 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-76dff48d59-d9gcd"] May 27 21:30:21.100424 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.100407 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-76dff48d59-d9gcd" May 27 21:30:21.103882 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.103866 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-vbpsr\"" May 27 21:30:21.104071 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.104059 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" May 27 21:30:21.104505 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.104485 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" May 27 21:30:21.104610 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.104597 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" May 27 21:30:21.104610 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.104601 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" May 27 21:30:21.109722 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.109702 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" May 27 21:30:21.111926 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.111906 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-76dff48d59-d9gcd"] May 27 21:30:21.132483 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.132461 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/16b8cf1e-1181-43fc-92e0-635b8523248d-trusted-ca\") pod \"console-operator-76dff48d59-d9gcd\" (UID: \"16b8cf1e-1181-43fc-92e0-635b8523248d\") " pod="openshift-console-operator/console-operator-76dff48d59-d9gcd" May 27 21:30:21.132569 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.132490 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/16b8cf1e-1181-43fc-92e0-635b8523248d-config\") pod \"console-operator-76dff48d59-d9gcd\" (UID: \"16b8cf1e-1181-43fc-92e0-635b8523248d\") " pod="openshift-console-operator/console-operator-76dff48d59-d9gcd" May 27 21:30:21.132569 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.132507 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dflgx\" (UniqueName: \"kubernetes.io/projected/16b8cf1e-1181-43fc-92e0-635b8523248d-kube-api-access-dflgx\") pod \"console-operator-76dff48d59-d9gcd\" (UID: \"16b8cf1e-1181-43fc-92e0-635b8523248d\") " pod="openshift-console-operator/console-operator-76dff48d59-d9gcd" May 27 21:30:21.132569 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.132527 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4rjxw\" (UniqueName: \"kubernetes.io/projected/3bdeafef-3082-4f0e-8add-fcdf60551c63-kube-api-access-4rjxw\") pod \"volume-data-source-validator-5cf8c89d64-c5dbz\" (UID: \"3bdeafef-3082-4f0e-8add-fcdf60551c63\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-c5dbz" May 27 21:30:21.132714 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.132703 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/16b8cf1e-1181-43fc-92e0-635b8523248d-serving-cert\") pod \"console-operator-76dff48d59-d9gcd\" (UID: \"16b8cf1e-1181-43fc-92e0-635b8523248d\") " pod="openshift-console-operator/console-operator-76dff48d59-d9gcd" May 27 21:30:21.141297 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.141255 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rjxw\" (UniqueName: \"kubernetes.io/projected/3bdeafef-3082-4f0e-8add-fcdf60551c63-kube-api-access-4rjxw\") pod \"volume-data-source-validator-5cf8c89d64-c5dbz\" (UID: \"3bdeafef-3082-4f0e-8add-fcdf60551c63\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-c5dbz" May 27 21:30:21.191961 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.191732 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-z76cl"] May 27 21:30:21.193636 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.193619 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-688f8b8554-fn4zx"] May 27 21:30:21.193785 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.193767 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-z76cl" May 27 21:30:21.195163 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.195144 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-688f8b8554-fn4zx" May 27 21:30:21.201401 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.201382 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" May 27 21:30:21.201401 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.201398 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" May 27 21:30:21.201575 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.201558 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" May 27 21:30:21.201575 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.201559 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-h9rsr\"" May 27 21:30:21.202019 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.202004 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-5qsfn\"" May 27 21:30:21.202098 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.202005 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" May 27 21:30:21.202098 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.202033 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" May 27 21:30:21.202639 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.202619 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" May 27 21:30:21.202709 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.202696 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" May 27 21:30:21.202794 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.202779 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" May 27 21:30:21.202859 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.202820 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" May 27 21:30:21.203655 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.203640 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" May 27 21:30:21.209890 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.209854 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-z76cl"] May 27 21:30:21.214253 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.214233 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-688f8b8554-fn4zx"] May 27 21:30:21.233435 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.233377 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1df7cdcd-3a57-4d1a-97d4-2638a9f49055-config\") pod \"kube-storage-version-migrator-operator-565644f49-z76cl\" (UID: \"1df7cdcd-3a57-4d1a-97d4-2638a9f49055\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-z76cl" May 27 21:30:21.233435 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.233405 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/51660acb-022f-4265-bec8-ae93f5315b17-default-certificate\") pod \"router-default-688f8b8554-fn4zx\" (UID: \"51660acb-022f-4265-bec8-ae93f5315b17\") " pod="openshift-ingress/router-default-688f8b8554-fn4zx" May 27 21:30:21.233543 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.233441 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gstlz\" (UniqueName: \"kubernetes.io/projected/1df7cdcd-3a57-4d1a-97d4-2638a9f49055-kube-api-access-gstlz\") pod \"kube-storage-version-migrator-operator-565644f49-z76cl\" (UID: \"1df7cdcd-3a57-4d1a-97d4-2638a9f49055\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-z76cl" May 27 21:30:21.233543 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.233474 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/16b8cf1e-1181-43fc-92e0-635b8523248d-trusted-ca\") pod \"console-operator-76dff48d59-d9gcd\" (UID: \"16b8cf1e-1181-43fc-92e0-635b8523248d\") " pod="openshift-console-operator/console-operator-76dff48d59-d9gcd" May 27 21:30:21.233543 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.233511 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/51660acb-022f-4265-bec8-ae93f5315b17-stats-auth\") pod \"router-default-688f8b8554-fn4zx\" (UID: \"51660acb-022f-4265-bec8-ae93f5315b17\") " pod="openshift-ingress/router-default-688f8b8554-fn4zx" May 27 21:30:21.233543 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.233534 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/16b8cf1e-1181-43fc-92e0-635b8523248d-config\") pod \"console-operator-76dff48d59-d9gcd\" (UID: \"16b8cf1e-1181-43fc-92e0-635b8523248d\") " pod="openshift-console-operator/console-operator-76dff48d59-d9gcd" May 27 21:30:21.233662 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.233594 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dflgx\" (UniqueName: \"kubernetes.io/projected/16b8cf1e-1181-43fc-92e0-635b8523248d-kube-api-access-dflgx\") pod \"console-operator-76dff48d59-d9gcd\" (UID: \"16b8cf1e-1181-43fc-92e0-635b8523248d\") " pod="openshift-console-operator/console-operator-76dff48d59-d9gcd" May 27 21:30:21.233662 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.233646 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/51660acb-022f-4265-bec8-ae93f5315b17-metrics-certs\") pod \"router-default-688f8b8554-fn4zx\" (UID: \"51660acb-022f-4265-bec8-ae93f5315b17\") " pod="openshift-ingress/router-default-688f8b8554-fn4zx" May 27 21:30:21.233743 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.233678 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/16b8cf1e-1181-43fc-92e0-635b8523248d-serving-cert\") pod \"console-operator-76dff48d59-d9gcd\" (UID: \"16b8cf1e-1181-43fc-92e0-635b8523248d\") " pod="openshift-console-operator/console-operator-76dff48d59-d9gcd" May 27 21:30:21.233743 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.233724 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1df7cdcd-3a57-4d1a-97d4-2638a9f49055-serving-cert\") pod \"kube-storage-version-migrator-operator-565644f49-z76cl\" (UID: \"1df7cdcd-3a57-4d1a-97d4-2638a9f49055\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-z76cl" May 27 21:30:21.233837 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.233789 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51660acb-022f-4265-bec8-ae93f5315b17-service-ca-bundle\") pod \"router-default-688f8b8554-fn4zx\" (UID: \"51660acb-022f-4265-bec8-ae93f5315b17\") " pod="openshift-ingress/router-default-688f8b8554-fn4zx" May 27 21:30:21.233880 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.233860 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lw4dj\" (UniqueName: \"kubernetes.io/projected/51660acb-022f-4265-bec8-ae93f5315b17-kube-api-access-lw4dj\") pod \"router-default-688f8b8554-fn4zx\" (UID: \"51660acb-022f-4265-bec8-ae93f5315b17\") " pod="openshift-ingress/router-default-688f8b8554-fn4zx" May 27 21:30:21.234390 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.234371 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/16b8cf1e-1181-43fc-92e0-635b8523248d-config\") pod \"console-operator-76dff48d59-d9gcd\" (UID: \"16b8cf1e-1181-43fc-92e0-635b8523248d\") " pod="openshift-console-operator/console-operator-76dff48d59-d9gcd" May 27 21:30:21.234747 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.234729 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/16b8cf1e-1181-43fc-92e0-635b8523248d-trusted-ca\") pod \"console-operator-76dff48d59-d9gcd\" (UID: \"16b8cf1e-1181-43fc-92e0-635b8523248d\") " pod="openshift-console-operator/console-operator-76dff48d59-d9gcd" May 27 21:30:21.235992 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.235978 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/16b8cf1e-1181-43fc-92e0-635b8523248d-serving-cert\") pod \"console-operator-76dff48d59-d9gcd\" (UID: \"16b8cf1e-1181-43fc-92e0-635b8523248d\") " pod="openshift-console-operator/console-operator-76dff48d59-d9gcd" May 27 21:30:21.260055 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.260028 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dflgx\" (UniqueName: \"kubernetes.io/projected/16b8cf1e-1181-43fc-92e0-635b8523248d-kube-api-access-dflgx\") pod \"console-operator-76dff48d59-d9gcd\" (UID: \"16b8cf1e-1181-43fc-92e0-635b8523248d\") " pod="openshift-console-operator/console-operator-76dff48d59-d9gcd" May 27 21:30:21.281331 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.281308 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-c5dbz" May 27 21:30:21.297557 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.297531 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7f44b6dd87-krplr"] May 27 21:30:21.315494 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.313599 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7f44b6dd87-krplr"] May 27 21:30:21.315494 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.313733 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-krplr" May 27 21:30:21.316504 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.316476 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-f5p49\"" May 27 21:30:21.334707 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.334682 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/51660acb-022f-4265-bec8-ae93f5315b17-stats-auth\") pod \"router-default-688f8b8554-fn4zx\" (UID: \"51660acb-022f-4265-bec8-ae93f5315b17\") " pod="openshift-ingress/router-default-688f8b8554-fn4zx" May 27 21:30:21.334822 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.334717 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/51660acb-022f-4265-bec8-ae93f5315b17-metrics-certs\") pod \"router-default-688f8b8554-fn4zx\" (UID: \"51660acb-022f-4265-bec8-ae93f5315b17\") " pod="openshift-ingress/router-default-688f8b8554-fn4zx" May 27 21:30:21.334822 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.334744 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhkvd\" (UniqueName: \"kubernetes.io/projected/beeba572-6c32-4e0b-b700-9e9271c0caef-kube-api-access-qhkvd\") pod \"network-check-source-7f44b6dd87-krplr\" (UID: \"beeba572-6c32-4e0b-b700-9e9271c0caef\") " pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-krplr" May 27 21:30:21.334822 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.334779 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1df7cdcd-3a57-4d1a-97d4-2638a9f49055-serving-cert\") pod \"kube-storage-version-migrator-operator-565644f49-z76cl\" (UID: \"1df7cdcd-3a57-4d1a-97d4-2638a9f49055\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-z76cl" May 27 21:30:21.334973 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:30:21.334821 2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found May 27 21:30:21.334973 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.334822 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51660acb-022f-4265-bec8-ae93f5315b17-service-ca-bundle\") pod \"router-default-688f8b8554-fn4zx\" (UID: \"51660acb-022f-4265-bec8-ae93f5315b17\") " pod="openshift-ingress/router-default-688f8b8554-fn4zx" May 27 21:30:21.334973 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:30:21.334875 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51660acb-022f-4265-bec8-ae93f5315b17-metrics-certs podName:51660acb-022f-4265-bec8-ae93f5315b17 nodeName:}" failed. No retries permitted until 2026-05-27 21:30:21.834857744 +0000 UTC m=+95.309062463 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/51660acb-022f-4265-bec8-ae93f5315b17-metrics-certs") pod "router-default-688f8b8554-fn4zx" (UID: "51660acb-022f-4265-bec8-ae93f5315b17") : secret "router-metrics-certs-default" not found May 27 21:30:21.334973 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:30:21.334913 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/51660acb-022f-4265-bec8-ae93f5315b17-service-ca-bundle podName:51660acb-022f-4265-bec8-ae93f5315b17 nodeName:}" failed. No retries permitted until 2026-05-27 21:30:21.834902645 +0000 UTC m=+95.309107366 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/51660acb-022f-4265-bec8-ae93f5315b17-service-ca-bundle") pod "router-default-688f8b8554-fn4zx" (UID: "51660acb-022f-4265-bec8-ae93f5315b17") : configmap references non-existent config key: service-ca.crt May 27 21:30:21.334973 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.334910 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lw4dj\" (UniqueName: \"kubernetes.io/projected/51660acb-022f-4265-bec8-ae93f5315b17-kube-api-access-lw4dj\") pod \"router-default-688f8b8554-fn4zx\" (UID: \"51660acb-022f-4265-bec8-ae93f5315b17\") " pod="openshift-ingress/router-default-688f8b8554-fn4zx" May 27 21:30:21.334973 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.334963 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1df7cdcd-3a57-4d1a-97d4-2638a9f49055-config\") pod \"kube-storage-version-migrator-operator-565644f49-z76cl\" (UID: \"1df7cdcd-3a57-4d1a-97d4-2638a9f49055\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-z76cl" May 27 21:30:21.335325 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.334989 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/51660acb-022f-4265-bec8-ae93f5315b17-default-certificate\") pod \"router-default-688f8b8554-fn4zx\" (UID: \"51660acb-022f-4265-bec8-ae93f5315b17\") " pod="openshift-ingress/router-default-688f8b8554-fn4zx" May 27 21:30:21.335325 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.335030 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gstlz\" (UniqueName: \"kubernetes.io/projected/1df7cdcd-3a57-4d1a-97d4-2638a9f49055-kube-api-access-gstlz\") pod \"kube-storage-version-migrator-operator-565644f49-z76cl\" (UID: \"1df7cdcd-3a57-4d1a-97d4-2638a9f49055\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-z76cl" May 27 21:30:21.335545 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.335528 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1df7cdcd-3a57-4d1a-97d4-2638a9f49055-config\") pod \"kube-storage-version-migrator-operator-565644f49-z76cl\" (UID: \"1df7cdcd-3a57-4d1a-97d4-2638a9f49055\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-z76cl" May 27 21:30:21.337422 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.337395 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1df7cdcd-3a57-4d1a-97d4-2638a9f49055-serving-cert\") pod \"kube-storage-version-migrator-operator-565644f49-z76cl\" (UID: \"1df7cdcd-3a57-4d1a-97d4-2638a9f49055\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-z76cl" May 27 21:30:21.337583 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.337568 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/51660acb-022f-4265-bec8-ae93f5315b17-stats-auth\") pod \"router-default-688f8b8554-fn4zx\" (UID: \"51660acb-022f-4265-bec8-ae93f5315b17\") " pod="openshift-ingress/router-default-688f8b8554-fn4zx" May 27 21:30:21.337715 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.337697 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/51660acb-022f-4265-bec8-ae93f5315b17-default-certificate\") pod \"router-default-688f8b8554-fn4zx\" (UID: \"51660acb-022f-4265-bec8-ae93f5315b17\") " pod="openshift-ingress/router-default-688f8b8554-fn4zx" May 27 21:30:21.355450 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.355413 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gstlz\" (UniqueName: \"kubernetes.io/projected/1df7cdcd-3a57-4d1a-97d4-2638a9f49055-kube-api-access-gstlz\") pod \"kube-storage-version-migrator-operator-565644f49-z76cl\" (UID: \"1df7cdcd-3a57-4d1a-97d4-2638a9f49055\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-z76cl" May 27 21:30:21.370741 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.370695 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lw4dj\" (UniqueName: \"kubernetes.io/projected/51660acb-022f-4265-bec8-ae93f5315b17-kube-api-access-lw4dj\") pod \"router-default-688f8b8554-fn4zx\" (UID: \"51660acb-022f-4265-bec8-ae93f5315b17\") " pod="openshift-ingress/router-default-688f8b8554-fn4zx" May 27 21:30:21.409285 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.409243 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-76dff48d59-d9gcd" May 27 21:30:21.434891 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:30:21.434860 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bdeafef_3082_4f0e_8add_fcdf60551c63.slice/crio-e8f681c6f7b01552376045e902b5e8fc58ff13af8cb9f91a3771ac2f4d1c67cf WatchSource:0}: Error finding container e8f681c6f7b01552376045e902b5e8fc58ff13af8cb9f91a3771ac2f4d1c67cf: Status 404 returned error can't find the container with id e8f681c6f7b01552376045e902b5e8fc58ff13af8cb9f91a3771ac2f4d1c67cf May 27 21:30:21.435784 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.435743 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-c5dbz"] May 27 21:30:21.436024 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.436002 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qhkvd\" (UniqueName: \"kubernetes.io/projected/beeba572-6c32-4e0b-b700-9e9271c0caef-kube-api-access-qhkvd\") pod \"network-check-source-7f44b6dd87-krplr\" (UID: \"beeba572-6c32-4e0b-b700-9e9271c0caef\") " pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-krplr" May 27 21:30:21.448818 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.448794 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhkvd\" (UniqueName: \"kubernetes.io/projected/beeba572-6c32-4e0b-b700-9e9271c0caef-kube-api-access-qhkvd\") pod \"network-check-source-7f44b6dd87-krplr\" (UID: \"beeba572-6c32-4e0b-b700-9e9271c0caef\") " pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-krplr" May 27 21:30:21.478368 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.478332 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-c5dbz" event={"ID":"3bdeafef-3082-4f0e-8add-fcdf60551c63","Type":"ContainerStarted","Data":"e8f681c6f7b01552376045e902b5e8fc58ff13af8cb9f91a3771ac2f4d1c67cf"} May 27 21:30:21.503666 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.503596 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-z76cl" May 27 21:30:21.532338 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.532302 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-76dff48d59-d9gcd"] May 27 21:30:21.534458 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:30:21.534427 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16b8cf1e_1181_43fc_92e0_635b8523248d.slice/crio-e3ad2dee7d79791ce6e3801c37553a3c3ba48ca8f154cadd6265bb0e4238471d WatchSource:0}: Error finding container e3ad2dee7d79791ce6e3801c37553a3c3ba48ca8f154cadd6265bb0e4238471d: Status 404 returned error can't find the container with id e3ad2dee7d79791ce6e3801c37553a3c3ba48ca8f154cadd6265bb0e4238471d May 27 21:30:21.624407 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.624130 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-krplr" May 27 21:30:21.626028 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.625659 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-z76cl"] May 27 21:30:21.629720 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:30:21.629696 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1df7cdcd_3a57_4d1a_97d4_2638a9f49055.slice/crio-dc2e48322b14e6c420c402ca118b65959f1eaceaaf3b4dff174019f155cfb2c8 WatchSource:0}: Error finding container dc2e48322b14e6c420c402ca118b65959f1eaceaaf3b4dff174019f155cfb2c8: Status 404 returned error can't find the container with id dc2e48322b14e6c420c402ca118b65959f1eaceaaf3b4dff174019f155cfb2c8 May 27 21:30:21.745038 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.745014 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7f44b6dd87-krplr"] May 27 21:30:21.747457 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:30:21.747433 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbeeba572_6c32_4e0b_b700_9e9271c0caef.slice/crio-967e28babe436007033e0fca179b668b0564cc85480d53f9eac3047a93836521 WatchSource:0}: Error finding container 967e28babe436007033e0fca179b668b0564cc85480d53f9eac3047a93836521: Status 404 returned error can't find the container with id 967e28babe436007033e0fca179b668b0564cc85480d53f9eac3047a93836521 May 27 21:30:21.838784 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.838739 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51660acb-022f-4265-bec8-ae93f5315b17-service-ca-bundle\") pod \"router-default-688f8b8554-fn4zx\" (UID: \"51660acb-022f-4265-bec8-ae93f5315b17\") " pod="openshift-ingress/router-default-688f8b8554-fn4zx" May 27 21:30:21.838936 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:21.838829 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/51660acb-022f-4265-bec8-ae93f5315b17-metrics-certs\") pod \"router-default-688f8b8554-fn4zx\" (UID: \"51660acb-022f-4265-bec8-ae93f5315b17\") " pod="openshift-ingress/router-default-688f8b8554-fn4zx" May 27 21:30:21.838936 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:30:21.838905 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/51660acb-022f-4265-bec8-ae93f5315b17-service-ca-bundle podName:51660acb-022f-4265-bec8-ae93f5315b17 nodeName:}" failed. No retries permitted until 2026-05-27 21:30:22.838888971 +0000 UTC m=+96.313093683 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/51660acb-022f-4265-bec8-ae93f5315b17-service-ca-bundle") pod "router-default-688f8b8554-fn4zx" (UID: "51660acb-022f-4265-bec8-ae93f5315b17") : configmap references non-existent config key: service-ca.crt May 27 21:30:21.839017 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:30:21.838948 2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found May 27 21:30:21.839017 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:30:21.838989 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51660acb-022f-4265-bec8-ae93f5315b17-metrics-certs podName:51660acb-022f-4265-bec8-ae93f5315b17 nodeName:}" failed. No retries permitted until 2026-05-27 21:30:22.838975401 +0000 UTC m=+96.313180108 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/51660acb-022f-4265-bec8-ae93f5315b17-metrics-certs") pod "router-default-688f8b8554-fn4zx" (UID: "51660acb-022f-4265-bec8-ae93f5315b17") : secret "router-metrics-certs-default" not found May 27 21:30:22.482511 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:22.482454 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-z76cl" event={"ID":"1df7cdcd-3a57-4d1a-97d4-2638a9f49055","Type":"ContainerStarted","Data":"dc2e48322b14e6c420c402ca118b65959f1eaceaaf3b4dff174019f155cfb2c8"} May 27 21:30:22.483998 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:22.483938 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-krplr" event={"ID":"beeba572-6c32-4e0b-b700-9e9271c0caef","Type":"ContainerStarted","Data":"4348fad35d41c907d8abc594e589b7168fb016b81990f1569669ca58a74bf7a5"} May 27 21:30:22.483998 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:22.483975 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-krplr" event={"ID":"beeba572-6c32-4e0b-b700-9e9271c0caef","Type":"ContainerStarted","Data":"967e28babe436007033e0fca179b668b0564cc85480d53f9eac3047a93836521"} May 27 21:30:22.485391 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:22.485360 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-76dff48d59-d9gcd" event={"ID":"16b8cf1e-1181-43fc-92e0-635b8523248d","Type":"ContainerStarted","Data":"e3ad2dee7d79791ce6e3801c37553a3c3ba48ca8f154cadd6265bb0e4238471d"} May 27 21:30:22.502409 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:22.501574 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-krplr" podStartSLOduration=1.5015586669999998 podStartE2EDuration="1.501558667s" podCreationTimestamp="2026-05-27 21:30:21 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-27 21:30:22.50069007 +0000 UTC m=+95.974894800" watchObservedRunningTime="2026-05-27 21:30:22.501558667 +0000 UTC m=+95.975763395" May 27 21:30:22.850229 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:22.849711 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/51660acb-022f-4265-bec8-ae93f5315b17-metrics-certs\") pod \"router-default-688f8b8554-fn4zx\" (UID: \"51660acb-022f-4265-bec8-ae93f5315b17\") " pod="openshift-ingress/router-default-688f8b8554-fn4zx" May 27 21:30:22.850229 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:22.849792 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51660acb-022f-4265-bec8-ae93f5315b17-service-ca-bundle\") pod \"router-default-688f8b8554-fn4zx\" (UID: \"51660acb-022f-4265-bec8-ae93f5315b17\") " pod="openshift-ingress/router-default-688f8b8554-fn4zx" May 27 21:30:22.850229 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:30:22.849921 2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found May 27 21:30:22.850229 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:30:22.849959 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/51660acb-022f-4265-bec8-ae93f5315b17-service-ca-bundle podName:51660acb-022f-4265-bec8-ae93f5315b17 nodeName:}" failed. No retries permitted until 2026-05-27 21:30:24.849939053 +0000 UTC m=+98.324143762 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/51660acb-022f-4265-bec8-ae93f5315b17-service-ca-bundle") pod "router-default-688f8b8554-fn4zx" (UID: "51660acb-022f-4265-bec8-ae93f5315b17") : configmap references non-existent config key: service-ca.crt May 27 21:30:22.850229 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:30:22.849990 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51660acb-022f-4265-bec8-ae93f5315b17-metrics-certs podName:51660acb-022f-4265-bec8-ae93f5315b17 nodeName:}" failed. No retries permitted until 2026-05-27 21:30:24.84997274 +0000 UTC m=+98.324177455 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/51660acb-022f-4265-bec8-ae93f5315b17-metrics-certs") pod "router-default-688f8b8554-fn4zx" (UID: "51660acb-022f-4265-bec8-ae93f5315b17") : secret "router-metrics-certs-default" not found May 27 21:30:23.489448 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:23.489393 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-c5dbz" event={"ID":"3bdeafef-3082-4f0e-8add-fcdf60551c63","Type":"ContainerStarted","Data":"452f4c19a76c6e83c94ca3215034231074c7c2963815fb90339a9a24943d6b36"} May 27 21:30:23.511087 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:23.511040 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-c5dbz" podStartSLOduration=1.9555828800000001 podStartE2EDuration="3.51102427s" podCreationTimestamp="2026-05-27 21:30:20 +0000 UTC" firstStartedPulling="2026-05-27 21:30:21.436987339 +0000 UTC m=+94.911192047" lastFinishedPulling="2026-05-27 21:30:22.992428727 +0000 UTC m=+96.466633437" observedRunningTime="2026-05-27 21:30:23.511010765 +0000 UTC m=+96.985215495" watchObservedRunningTime="2026-05-27 21:30:23.51102427 +0000 UTC m=+96.985229000" May 27 21:30:23.758258 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:23.758162 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f0c8b7dd-e25c-44d5-a1ac-52e4cef19011-metrics-tls\") pod \"dns-default-4t29f\" (UID: \"f0c8b7dd-e25c-44d5-a1ac-52e4cef19011\") " pod="openshift-dns/dns-default-4t29f" May 27 21:30:23.758448 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:30:23.758371 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 27 21:30:23.758514 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:30:23.758449 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0c8b7dd-e25c-44d5-a1ac-52e4cef19011-metrics-tls podName:f0c8b7dd-e25c-44d5-a1ac-52e4cef19011 nodeName:}" failed. No retries permitted until 2026-05-27 21:31:27.758429125 +0000 UTC m=+161.232633849 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f0c8b7dd-e25c-44d5-a1ac-52e4cef19011-metrics-tls") pod "dns-default-4t29f" (UID: "f0c8b7dd-e25c-44d5-a1ac-52e4cef19011") : secret "dns-default-metrics-tls" not found May 27 21:30:23.859236 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:23.859194 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0a23d452-20fc-4665-9d3a-ef17d210a8d9-cert\") pod \"ingress-canary-5ntj8\" (UID: \"0a23d452-20fc-4665-9d3a-ef17d210a8d9\") " pod="openshift-ingress-canary/ingress-canary-5ntj8" May 27 21:30:23.859437 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:30:23.859331 2564 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 27 21:30:23.859437 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:30:23.859398 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a23d452-20fc-4665-9d3a-ef17d210a8d9-cert podName:0a23d452-20fc-4665-9d3a-ef17d210a8d9 nodeName:}" failed. No retries permitted until 2026-05-27 21:31:27.859380564 +0000 UTC m=+161.333585274 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/0a23d452-20fc-4665-9d3a-ef17d210a8d9-cert") pod "ingress-canary-5ntj8" (UID: "0a23d452-20fc-4665-9d3a-ef17d210a8d9") : secret "canary-serving-cert" not found May 27 21:30:24.492313 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:24.492278 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-z76cl" event={"ID":"1df7cdcd-3a57-4d1a-97d4-2638a9f49055","Type":"ContainerStarted","Data":"615a4c170ae95094dee992d7973e02dabbadd3b1625eeddfb2bb31802e0716bc"} May 27 21:30:24.493698 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:24.493676 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-d9gcd_16b8cf1e-1181-43fc-92e0-635b8523248d/console-operator/0.log" May 27 21:30:24.493793 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:24.493718 2564 generic.go:358] "Generic (PLEG): container finished" podID="16b8cf1e-1181-43fc-92e0-635b8523248d" containerID="ee568f21ea09e4925231efe0f1da2b10bcf5bf5090e69b37b56890df11c72c37" exitCode=255 May 27 21:30:24.493841 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:24.493808 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-76dff48d59-d9gcd" event={"ID":"16b8cf1e-1181-43fc-92e0-635b8523248d","Type":"ContainerDied","Data":"ee568f21ea09e4925231efe0f1da2b10bcf5bf5090e69b37b56890df11c72c37"} May 27 21:30:24.493969 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:24.493955 2564 scope.go:117] "RemoveContainer" containerID="ee568f21ea09e4925231efe0f1da2b10bcf5bf5090e69b37b56890df11c72c37" May 27 21:30:24.512855 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:24.512803 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-z76cl" podStartSLOduration=1.083541795 podStartE2EDuration="3.512789512s" podCreationTimestamp="2026-05-27 21:30:21 +0000 UTC" firstStartedPulling="2026-05-27 21:30:21.632575267 +0000 UTC m=+95.106779975" lastFinishedPulling="2026-05-27 21:30:24.061822982 +0000 UTC m=+97.536027692" observedRunningTime="2026-05-27 21:30:24.511197936 +0000 UTC m=+97.985402856" watchObservedRunningTime="2026-05-27 21:30:24.512789512 +0000 UTC m=+97.986994240" May 27 21:30:24.868135 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:24.868034 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51660acb-022f-4265-bec8-ae93f5315b17-service-ca-bundle\") pod \"router-default-688f8b8554-fn4zx\" (UID: \"51660acb-022f-4265-bec8-ae93f5315b17\") " pod="openshift-ingress/router-default-688f8b8554-fn4zx" May 27 21:30:24.868360 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:24.868150 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/51660acb-022f-4265-bec8-ae93f5315b17-metrics-certs\") pod \"router-default-688f8b8554-fn4zx\" (UID: \"51660acb-022f-4265-bec8-ae93f5315b17\") " pod="openshift-ingress/router-default-688f8b8554-fn4zx" May 27 21:30:24.868360 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:30:24.868218 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/51660acb-022f-4265-bec8-ae93f5315b17-service-ca-bundle podName:51660acb-022f-4265-bec8-ae93f5315b17 nodeName:}" failed. No retries permitted until 2026-05-27 21:30:28.868195204 +0000 UTC m=+102.342399924 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/51660acb-022f-4265-bec8-ae93f5315b17-service-ca-bundle") pod "router-default-688f8b8554-fn4zx" (UID: "51660acb-022f-4265-bec8-ae93f5315b17") : configmap references non-existent config key: service-ca.crt May 27 21:30:24.868360 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:30:24.868312 2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found May 27 21:30:24.868527 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:30:24.868372 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51660acb-022f-4265-bec8-ae93f5315b17-metrics-certs podName:51660acb-022f-4265-bec8-ae93f5315b17 nodeName:}" failed. No retries permitted until 2026-05-27 21:30:28.868355744 +0000 UTC m=+102.342560467 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/51660acb-022f-4265-bec8-ae93f5315b17-metrics-certs") pod "router-default-688f8b8554-fn4zx" (UID: "51660acb-022f-4265-bec8-ae93f5315b17") : secret "router-metrics-certs-default" not found May 27 21:30:25.501485 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:25.501455 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-d9gcd_16b8cf1e-1181-43fc-92e0-635b8523248d/console-operator/1.log" May 27 21:30:25.501889 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:25.501833 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-d9gcd_16b8cf1e-1181-43fc-92e0-635b8523248d/console-operator/0.log" May 27 21:30:25.501889 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:25.501866 2564 generic.go:358] "Generic (PLEG): container finished" podID="16b8cf1e-1181-43fc-92e0-635b8523248d" containerID="4877e2f8518334f6ff6eabc9329f05c3d8c46029332734a43724218c2732f250" exitCode=255 May 27 21:30:25.501996 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:25.501969 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-76dff48d59-d9gcd" event={"ID":"16b8cf1e-1181-43fc-92e0-635b8523248d","Type":"ContainerDied","Data":"4877e2f8518334f6ff6eabc9329f05c3d8c46029332734a43724218c2732f250"} May 27 21:30:25.502048 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:25.502034 2564 scope.go:117] "RemoveContainer" containerID="ee568f21ea09e4925231efe0f1da2b10bcf5bf5090e69b37b56890df11c72c37" May 27 21:30:25.502543 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:25.502330 2564 scope.go:117] "RemoveContainer" containerID="4877e2f8518334f6ff6eabc9329f05c3d8c46029332734a43724218c2732f250" May 27 21:30:25.502543 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:30:25.502481 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-76dff48d59-d9gcd_openshift-console-operator(16b8cf1e-1181-43fc-92e0-635b8523248d)\"" pod="openshift-console-operator/console-operator-76dff48d59-d9gcd" podUID="16b8cf1e-1181-43fc-92e0-635b8523248d" May 27 21:30:26.432949 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:26.432920 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-2kzbj" May 27 21:30:26.505608 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:26.505578 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-d9gcd_16b8cf1e-1181-43fc-92e0-635b8523248d/console-operator/1.log" May 27 21:30:26.505961 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:26.505892 2564 scope.go:117] "RemoveContainer" containerID="4877e2f8518334f6ff6eabc9329f05c3d8c46029332734a43724218c2732f250" May 27 21:30:26.506072 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:30:26.506055 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-76dff48d59-d9gcd_openshift-console-operator(16b8cf1e-1181-43fc-92e0-635b8523248d)\"" pod="openshift-console-operator/console-operator-76dff48d59-d9gcd" podUID="16b8cf1e-1181-43fc-92e0-635b8523248d" May 27 21:30:27.354336 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:27.354306 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-sc6tb_d895055d-d1d4-44a6-9024-8b244e769d5e/dns-node-resolver/0.log" May 27 21:30:28.353824 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:28.353801 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mdd25_e176311e-f128-4137-904b-7dd6fea5eed1/node-ca/0.log" May 27 21:30:28.573853 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:28.573819 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-67d667c7d-qzfwz"] May 27 21:30:28.576635 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:28.576620 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-67d667c7d-qzfwz" May 27 21:30:28.579237 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:28.579217 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" May 27 21:30:28.580372 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:28.580352 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" May 27 21:30:28.580486 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:28.580356 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" May 27 21:30:28.580486 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:28.580356 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" May 27 21:30:28.580486 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:28.580425 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-vfsvk\"" May 27 21:30:28.584705 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:28.584684 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-67d667c7d-qzfwz"] May 27 21:30:28.696836 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:28.696807 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/846a91a5-80ee-4c45-b35a-6b581b9ab53c-signing-key\") pod \"service-ca-67d667c7d-qzfwz\" (UID: \"846a91a5-80ee-4c45-b35a-6b581b9ab53c\") " pod="openshift-service-ca/service-ca-67d667c7d-qzfwz" May 27 21:30:28.697012 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:28.696864 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/846a91a5-80ee-4c45-b35a-6b581b9ab53c-signing-cabundle\") pod \"service-ca-67d667c7d-qzfwz\" (UID: \"846a91a5-80ee-4c45-b35a-6b581b9ab53c\") " pod="openshift-service-ca/service-ca-67d667c7d-qzfwz" May 27 21:30:28.697012 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:28.696989 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j55s5\" (UniqueName: \"kubernetes.io/projected/846a91a5-80ee-4c45-b35a-6b581b9ab53c-kube-api-access-j55s5\") pod \"service-ca-67d667c7d-qzfwz\" (UID: \"846a91a5-80ee-4c45-b35a-6b581b9ab53c\") " pod="openshift-service-ca/service-ca-67d667c7d-qzfwz" May 27 21:30:28.797455 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:28.797418 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/846a91a5-80ee-4c45-b35a-6b581b9ab53c-signing-key\") pod \"service-ca-67d667c7d-qzfwz\" (UID: \"846a91a5-80ee-4c45-b35a-6b581b9ab53c\") " pod="openshift-service-ca/service-ca-67d667c7d-qzfwz" May 27 21:30:28.797619 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:28.797486 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/846a91a5-80ee-4c45-b35a-6b581b9ab53c-signing-cabundle\") pod \"service-ca-67d667c7d-qzfwz\" (UID: \"846a91a5-80ee-4c45-b35a-6b581b9ab53c\") " pod="openshift-service-ca/service-ca-67d667c7d-qzfwz" May 27 21:30:28.797619 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:28.797550 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-j55s5\" (UniqueName: \"kubernetes.io/projected/846a91a5-80ee-4c45-b35a-6b581b9ab53c-kube-api-access-j55s5\") pod \"service-ca-67d667c7d-qzfwz\" (UID: \"846a91a5-80ee-4c45-b35a-6b581b9ab53c\") " pod="openshift-service-ca/service-ca-67d667c7d-qzfwz" May 27 21:30:28.798125 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:28.798104 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/846a91a5-80ee-4c45-b35a-6b581b9ab53c-signing-cabundle\") pod \"service-ca-67d667c7d-qzfwz\" (UID: \"846a91a5-80ee-4c45-b35a-6b581b9ab53c\") " pod="openshift-service-ca/service-ca-67d667c7d-qzfwz" May 27 21:30:28.799759 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:28.799743 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/846a91a5-80ee-4c45-b35a-6b581b9ab53c-signing-key\") pod \"service-ca-67d667c7d-qzfwz\" (UID: \"846a91a5-80ee-4c45-b35a-6b581b9ab53c\") " pod="openshift-service-ca/service-ca-67d667c7d-qzfwz" May 27 21:30:28.807123 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:28.807102 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j55s5\" (UniqueName: \"kubernetes.io/projected/846a91a5-80ee-4c45-b35a-6b581b9ab53c-kube-api-access-j55s5\") pod \"service-ca-67d667c7d-qzfwz\" (UID: \"846a91a5-80ee-4c45-b35a-6b581b9ab53c\") " pod="openshift-service-ca/service-ca-67d667c7d-qzfwz" May 27 21:30:28.884991 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:28.884962 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-67d667c7d-qzfwz" May 27 21:30:28.898813 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:28.898787 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/51660acb-022f-4265-bec8-ae93f5315b17-metrics-certs\") pod \"router-default-688f8b8554-fn4zx\" (UID: \"51660acb-022f-4265-bec8-ae93f5315b17\") " pod="openshift-ingress/router-default-688f8b8554-fn4zx" May 27 21:30:28.898925 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:28.898848 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51660acb-022f-4265-bec8-ae93f5315b17-service-ca-bundle\") pod \"router-default-688f8b8554-fn4zx\" (UID: \"51660acb-022f-4265-bec8-ae93f5315b17\") " pod="openshift-ingress/router-default-688f8b8554-fn4zx" May 27 21:30:28.898973 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:30:28.898935 2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found May 27 21:30:28.899033 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:30:28.898999 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/51660acb-022f-4265-bec8-ae93f5315b17-service-ca-bundle podName:51660acb-022f-4265-bec8-ae93f5315b17 nodeName:}" failed. No retries permitted until 2026-05-27 21:30:36.89898505 +0000 UTC m=+110.373189761 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/51660acb-022f-4265-bec8-ae93f5315b17-service-ca-bundle") pod "router-default-688f8b8554-fn4zx" (UID: "51660acb-022f-4265-bec8-ae93f5315b17") : configmap references non-existent config key: service-ca.crt May 27 21:30:28.899033 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:30:28.899016 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51660acb-022f-4265-bec8-ae93f5315b17-metrics-certs podName:51660acb-022f-4265-bec8-ae93f5315b17 nodeName:}" failed. No retries permitted until 2026-05-27 21:30:36.89901048 +0000 UTC m=+110.373215187 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/51660acb-022f-4265-bec8-ae93f5315b17-metrics-certs") pod "router-default-688f8b8554-fn4zx" (UID: "51660acb-022f-4265-bec8-ae93f5315b17") : secret "router-metrics-certs-default" not found May 27 21:30:29.000821 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:29.000779 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-67d667c7d-qzfwz"] May 27 21:30:29.003773 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:30:29.003748 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod846a91a5_80ee_4c45_b35a_6b581b9ab53c.slice/crio-2fa3586ba786bd8f77029c3d0929626c780f1fc40ca1e41f6a3d3701cbfaa657 WatchSource:0}: Error finding container 2fa3586ba786bd8f77029c3d0929626c780f1fc40ca1e41f6a3d3701cbfaa657: Status 404 returned error can't find the container with id 2fa3586ba786bd8f77029c3d0929626c780f1fc40ca1e41f6a3d3701cbfaa657 May 27 21:30:29.512330 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:29.512294 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-67d667c7d-qzfwz" event={"ID":"846a91a5-80ee-4c45-b35a-6b581b9ab53c","Type":"ContainerStarted","Data":"2fa3586ba786bd8f77029c3d0929626c780f1fc40ca1e41f6a3d3701cbfaa657"} May 27 21:30:31.410143 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:31.410108 2564 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-76dff48d59-d9gcd" May 27 21:30:31.410143 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:31.410143 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-76dff48d59-d9gcd" May 27 21:30:31.410680 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:31.410468 2564 scope.go:117] "RemoveContainer" containerID="4877e2f8518334f6ff6eabc9329f05c3d8c46029332734a43724218c2732f250" May 27 21:30:31.410680 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:30:31.410628 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-76dff48d59-d9gcd_openshift-console-operator(16b8cf1e-1181-43fc-92e0-635b8523248d)\"" pod="openshift-console-operator/console-operator-76dff48d59-d9gcd" podUID="16b8cf1e-1181-43fc-92e0-635b8523248d" May 27 21:30:31.518633 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:31.518595 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-67d667c7d-qzfwz" event={"ID":"846a91a5-80ee-4c45-b35a-6b581b9ab53c","Type":"ContainerStarted","Data":"2db9b99e5d7ba41077414c67c00b179438634b0595b8c191e962de44c313ffa0"} May 27 21:30:31.545198 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:31.545143 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-service-ca/service-ca-67d667c7d-qzfwz" podStartSLOduration=1.6442256020000001 podStartE2EDuration="3.545125828s" podCreationTimestamp="2026-05-27 21:30:28 +0000 UTC" firstStartedPulling="2026-05-27 21:30:29.005563069 +0000 UTC m=+102.479767776" lastFinishedPulling="2026-05-27 21:30:30.906463275 +0000 UTC m=+104.380668002" observedRunningTime="2026-05-27 21:30:31.543975665 +0000 UTC m=+105.018180394" watchObservedRunningTime="2026-05-27 21:30:31.545125828 +0000 UTC m=+105.019330557" May 27 21:30:36.963625 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:36.963585 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/51660acb-022f-4265-bec8-ae93f5315b17-metrics-certs\") pod \"router-default-688f8b8554-fn4zx\" (UID: \"51660acb-022f-4265-bec8-ae93f5315b17\") " pod="openshift-ingress/router-default-688f8b8554-fn4zx" May 27 21:30:36.964024 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:36.963646 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51660acb-022f-4265-bec8-ae93f5315b17-service-ca-bundle\") pod \"router-default-688f8b8554-fn4zx\" (UID: \"51660acb-022f-4265-bec8-ae93f5315b17\") " pod="openshift-ingress/router-default-688f8b8554-fn4zx" May 27 21:30:36.964024 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:30:36.963735 2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found May 27 21:30:36.964024 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:30:36.963768 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/51660acb-022f-4265-bec8-ae93f5315b17-service-ca-bundle podName:51660acb-022f-4265-bec8-ae93f5315b17 nodeName:}" failed. No retries permitted until 2026-05-27 21:30:52.963754224 +0000 UTC m=+126.437958931 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/51660acb-022f-4265-bec8-ae93f5315b17-service-ca-bundle") pod "router-default-688f8b8554-fn4zx" (UID: "51660acb-022f-4265-bec8-ae93f5315b17") : configmap references non-existent config key: service-ca.crt May 27 21:30:36.964024 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:30:36.963802 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51660acb-022f-4265-bec8-ae93f5315b17-metrics-certs podName:51660acb-022f-4265-bec8-ae93f5315b17 nodeName:}" failed. No retries permitted until 2026-05-27 21:30:52.963788762 +0000 UTC m=+126.437993470 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/51660acb-022f-4265-bec8-ae93f5315b17-metrics-certs") pod "router-default-688f8b8554-fn4zx" (UID: "51660acb-022f-4265-bec8-ae93f5315b17") : secret "router-metrics-certs-default" not found May 27 21:30:46.079717 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:46.079677 2564 scope.go:117] "RemoveContainer" containerID="4877e2f8518334f6ff6eabc9329f05c3d8c46029332734a43724218c2732f250" May 27 21:30:46.553796 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:46.553770 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-d9gcd_16b8cf1e-1181-43fc-92e0-635b8523248d/console-operator/2.log" May 27 21:30:46.554152 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:46.554137 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-d9gcd_16b8cf1e-1181-43fc-92e0-635b8523248d/console-operator/1.log" May 27 21:30:46.554207 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:46.554172 2564 generic.go:358] "Generic (PLEG): container finished" podID="16b8cf1e-1181-43fc-92e0-635b8523248d" containerID="62ba8701691a698f432ece42c92493f5e343140dadc33cb2f8bf9afcf7556e23" exitCode=255 May 27 21:30:46.554249 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:46.554234 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-76dff48d59-d9gcd" event={"ID":"16b8cf1e-1181-43fc-92e0-635b8523248d","Type":"ContainerDied","Data":"62ba8701691a698f432ece42c92493f5e343140dadc33cb2f8bf9afcf7556e23"} May 27 21:30:46.554310 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:46.554293 2564 scope.go:117] "RemoveContainer" containerID="4877e2f8518334f6ff6eabc9329f05c3d8c46029332734a43724218c2732f250" May 27 21:30:46.554600 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:46.554580 2564 scope.go:117] "RemoveContainer" containerID="62ba8701691a698f432ece42c92493f5e343140dadc33cb2f8bf9afcf7556e23" May 27 21:30:46.554764 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:30:46.554748 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-76dff48d59-d9gcd_openshift-console-operator(16b8cf1e-1181-43fc-92e0-635b8523248d)\"" pod="openshift-console-operator/console-operator-76dff48d59-d9gcd" podUID="16b8cf1e-1181-43fc-92e0-635b8523248d" May 27 21:30:47.558343 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:47.558315 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-d9gcd_16b8cf1e-1181-43fc-92e0-635b8523248d/console-operator/2.log" May 27 21:30:51.409405 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:51.409352 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-76dff48d59-d9gcd" May 27 21:30:51.409405 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:51.409410 2564 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-76dff48d59-d9gcd" May 27 21:30:51.409842 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:51.409740 2564 scope.go:117] "RemoveContainer" containerID="62ba8701691a698f432ece42c92493f5e343140dadc33cb2f8bf9afcf7556e23" May 27 21:30:51.409933 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:30:51.409914 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-76dff48d59-d9gcd_openshift-console-operator(16b8cf1e-1181-43fc-92e0-635b8523248d)\"" pod="openshift-console-operator/console-operator-76dff48d59-d9gcd" podUID="16b8cf1e-1181-43fc-92e0-635b8523248d" May 27 21:30:52.990420 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:52.990380 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/51660acb-022f-4265-bec8-ae93f5315b17-metrics-certs\") pod \"router-default-688f8b8554-fn4zx\" (UID: \"51660acb-022f-4265-bec8-ae93f5315b17\") " pod="openshift-ingress/router-default-688f8b8554-fn4zx" May 27 21:30:52.990792 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:52.990445 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51660acb-022f-4265-bec8-ae93f5315b17-service-ca-bundle\") pod \"router-default-688f8b8554-fn4zx\" (UID: \"51660acb-022f-4265-bec8-ae93f5315b17\") " pod="openshift-ingress/router-default-688f8b8554-fn4zx" May 27 21:30:52.991007 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:52.990985 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51660acb-022f-4265-bec8-ae93f5315b17-service-ca-bundle\") pod \"router-default-688f8b8554-fn4zx\" (UID: \"51660acb-022f-4265-bec8-ae93f5315b17\") " pod="openshift-ingress/router-default-688f8b8554-fn4zx" May 27 21:30:52.992623 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:52.992606 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/51660acb-022f-4265-bec8-ae93f5315b17-metrics-certs\") pod \"router-default-688f8b8554-fn4zx\" (UID: \"51660acb-022f-4265-bec8-ae93f5315b17\") " pod="openshift-ingress/router-default-688f8b8554-fn4zx" May 27 21:30:53.010207 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:53.010180 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-5qsfn\"" May 27 21:30:53.018353 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:53.018336 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-688f8b8554-fn4zx" May 27 21:30:53.142752 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:53.142717 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-688f8b8554-fn4zx"] May 27 21:30:53.145940 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:30:53.145913 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51660acb_022f_4265_bec8_ae93f5315b17.slice/crio-f5b986c7427d0a032decd1637781257fa9a84fb04c8af832d84fafb7f072ab9b WatchSource:0}: Error finding container f5b986c7427d0a032decd1637781257fa9a84fb04c8af832d84fafb7f072ab9b: Status 404 returned error can't find the container with id f5b986c7427d0a032decd1637781257fa9a84fb04c8af832d84fafb7f072ab9b May 27 21:30:53.573669 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:53.573582 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-688f8b8554-fn4zx" event={"ID":"51660acb-022f-4265-bec8-ae93f5315b17","Type":"ContainerStarted","Data":"f44eb4b39c678de6b93463df77c2aaa3ef7f64b1f50e255274076bc231b4c1f2"} May 27 21:30:53.573669 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:53.573616 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-688f8b8554-fn4zx" event={"ID":"51660acb-022f-4265-bec8-ae93f5315b17","Type":"ContainerStarted","Data":"f5b986c7427d0a032decd1637781257fa9a84fb04c8af832d84fafb7f072ab9b"} May 27 21:30:53.593777 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:53.593730 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/router-default-688f8b8554-fn4zx" podStartSLOduration=32.59371717 podStartE2EDuration="32.59371717s" podCreationTimestamp="2026-05-27 21:30:21 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-27 21:30:53.593420119 +0000 UTC m=+127.067624848" watchObservedRunningTime="2026-05-27 21:30:53.59371717 +0000 UTC m=+127.067921895" May 27 21:30:54.019574 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:54.019520 2564 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-688f8b8554-fn4zx" May 27 21:30:54.022066 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:54.022044 2564 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-688f8b8554-fn4zx" May 27 21:30:54.575818 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:54.575793 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-688f8b8554-fn4zx" May 27 21:30:54.577041 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:54.577022 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-688f8b8554-fn4zx" May 27 21:30:55.082933 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.082905 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-7wjhw"] May 27 21:30:55.085518 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.085499 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7wjhw" May 27 21:30:55.088009 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.087986 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" May 27 21:30:55.089055 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.089029 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" May 27 21:30:55.089185 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.089034 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" May 27 21:30:55.089185 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.089037 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" May 27 21:30:55.089185 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.089103 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8gwtp\"" May 27 21:30:55.098647 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.098622 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7wjhw"] May 27 21:30:55.187252 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.187214 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7c87c4fbbc-psmsw"] May 27 21:30:55.189194 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.189173 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c87c4fbbc-psmsw" May 27 21:30:55.191802 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.191780 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" May 27 21:30:55.191918 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.191817 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" May 27 21:30:55.191918 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.191909 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-8rshs\"" May 27 21:30:55.192147 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.192130 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" May 27 21:30:55.199469 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.199445 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" May 27 21:30:55.207631 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.207609 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aafa4c59-c141-4e77-ae60-d6e48d468223-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7wjhw\" (UID: \"aafa4c59-c141-4e77-ae60-d6e48d468223\") " pod="openshift-insights/insights-runtime-extractor-7wjhw" May 27 21:30:55.207734 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.207715 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/aafa4c59-c141-4e77-ae60-d6e48d468223-crio-socket\") pod \"insights-runtime-extractor-7wjhw\" (UID: \"aafa4c59-c141-4e77-ae60-d6e48d468223\") " pod="openshift-insights/insights-runtime-extractor-7wjhw" May 27 21:30:55.207775 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.207765 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mm292\" (UniqueName: \"kubernetes.io/projected/aafa4c59-c141-4e77-ae60-d6e48d468223-kube-api-access-mm292\") pod \"insights-runtime-extractor-7wjhw\" (UID: \"aafa4c59-c141-4e77-ae60-d6e48d468223\") " pod="openshift-insights/insights-runtime-extractor-7wjhw" May 27 21:30:55.207827 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.207813 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/aafa4c59-c141-4e77-ae60-d6e48d468223-data-volume\") pod \"insights-runtime-extractor-7wjhw\" (UID: \"aafa4c59-c141-4e77-ae60-d6e48d468223\") " pod="openshift-insights/insights-runtime-extractor-7wjhw" May 27 21:30:55.207869 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.207842 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aafa4c59-c141-4e77-ae60-d6e48d468223-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7wjhw\" (UID: \"aafa4c59-c141-4e77-ae60-d6e48d468223\") " pod="openshift-insights/insights-runtime-extractor-7wjhw" May 27 21:30:55.213429 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.213407 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7c87c4fbbc-psmsw"] May 27 21:30:55.308730 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.308692 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mm292\" (UniqueName: \"kubernetes.io/projected/aafa4c59-c141-4e77-ae60-d6e48d468223-kube-api-access-mm292\") pod \"insights-runtime-extractor-7wjhw\" (UID: \"aafa4c59-c141-4e77-ae60-d6e48d468223\") " pod="openshift-insights/insights-runtime-extractor-7wjhw" May 27 21:30:55.308907 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.308756 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8nms\" (UniqueName: \"kubernetes.io/projected/d78abc3c-065a-43d7-a5e0-3cc662c0d057-kube-api-access-n8nms\") pod \"image-registry-7c87c4fbbc-psmsw\" (UID: \"d78abc3c-065a-43d7-a5e0-3cc662c0d057\") " pod="openshift-image-registry/image-registry-7c87c4fbbc-psmsw" May 27 21:30:55.308907 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.308819 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d78abc3c-065a-43d7-a5e0-3cc662c0d057-image-registry-private-configuration\") pod \"image-registry-7c87c4fbbc-psmsw\" (UID: \"d78abc3c-065a-43d7-a5e0-3cc662c0d057\") " pod="openshift-image-registry/image-registry-7c87c4fbbc-psmsw" May 27 21:30:55.308907 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.308899 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aafa4c59-c141-4e77-ae60-d6e48d468223-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7wjhw\" (UID: \"aafa4c59-c141-4e77-ae60-d6e48d468223\") " pod="openshift-insights/insights-runtime-extractor-7wjhw" May 27 21:30:55.309074 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.308923 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d78abc3c-065a-43d7-a5e0-3cc662c0d057-bound-sa-token\") pod \"image-registry-7c87c4fbbc-psmsw\" (UID: \"d78abc3c-065a-43d7-a5e0-3cc662c0d057\") " pod="openshift-image-registry/image-registry-7c87c4fbbc-psmsw" May 27 21:30:55.309074 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.308954 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d78abc3c-065a-43d7-a5e0-3cc662c0d057-installation-pull-secrets\") pod \"image-registry-7c87c4fbbc-psmsw\" (UID: \"d78abc3c-065a-43d7-a5e0-3cc662c0d057\") " pod="openshift-image-registry/image-registry-7c87c4fbbc-psmsw" May 27 21:30:55.309074 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.308975 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aafa4c59-c141-4e77-ae60-d6e48d468223-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7wjhw\" (UID: \"aafa4c59-c141-4e77-ae60-d6e48d468223\") " pod="openshift-insights/insights-runtime-extractor-7wjhw" May 27 21:30:55.309074 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.309036 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/aafa4c59-c141-4e77-ae60-d6e48d468223-crio-socket\") pod \"insights-runtime-extractor-7wjhw\" (UID: \"aafa4c59-c141-4e77-ae60-d6e48d468223\") " pod="openshift-insights/insights-runtime-extractor-7wjhw" May 27 21:30:55.309343 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.309081 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/aafa4c59-c141-4e77-ae60-d6e48d468223-data-volume\") pod \"insights-runtime-extractor-7wjhw\" (UID: \"aafa4c59-c141-4e77-ae60-d6e48d468223\") " pod="openshift-insights/insights-runtime-extractor-7wjhw" May 27 21:30:55.309343 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.309112 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d78abc3c-065a-43d7-a5e0-3cc662c0d057-ca-trust-extracted\") pod \"image-registry-7c87c4fbbc-psmsw\" (UID: \"d78abc3c-065a-43d7-a5e0-3cc662c0d057\") " pod="openshift-image-registry/image-registry-7c87c4fbbc-psmsw" May 27 21:30:55.309343 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.309145 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d78abc3c-065a-43d7-a5e0-3cc662c0d057-registry-certificates\") pod \"image-registry-7c87c4fbbc-psmsw\" (UID: \"d78abc3c-065a-43d7-a5e0-3cc662c0d057\") " pod="openshift-image-registry/image-registry-7c87c4fbbc-psmsw" May 27 21:30:55.309343 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.309158 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/aafa4c59-c141-4e77-ae60-d6e48d468223-crio-socket\") pod \"insights-runtime-extractor-7wjhw\" (UID: \"aafa4c59-c141-4e77-ae60-d6e48d468223\") " pod="openshift-insights/insights-runtime-extractor-7wjhw" May 27 21:30:55.309343 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.309177 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d78abc3c-065a-43d7-a5e0-3cc662c0d057-trusted-ca\") pod \"image-registry-7c87c4fbbc-psmsw\" (UID: \"d78abc3c-065a-43d7-a5e0-3cc662c0d057\") " pod="openshift-image-registry/image-registry-7c87c4fbbc-psmsw" May 27 21:30:55.309343 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.309252 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d78abc3c-065a-43d7-a5e0-3cc662c0d057-registry-tls\") pod \"image-registry-7c87c4fbbc-psmsw\" (UID: \"d78abc3c-065a-43d7-a5e0-3cc662c0d057\") " pod="openshift-image-registry/image-registry-7c87c4fbbc-psmsw" May 27 21:30:55.309564 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.309473 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/aafa4c59-c141-4e77-ae60-d6e48d468223-data-volume\") pod \"insights-runtime-extractor-7wjhw\" (UID: \"aafa4c59-c141-4e77-ae60-d6e48d468223\") " pod="openshift-insights/insights-runtime-extractor-7wjhw" May 27 21:30:55.309624 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.309608 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aafa4c59-c141-4e77-ae60-d6e48d468223-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7wjhw\" (UID: \"aafa4c59-c141-4e77-ae60-d6e48d468223\") " pod="openshift-insights/insights-runtime-extractor-7wjhw" May 27 21:30:55.311291 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.311256 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aafa4c59-c141-4e77-ae60-d6e48d468223-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7wjhw\" (UID: \"aafa4c59-c141-4e77-ae60-d6e48d468223\") " pod="openshift-insights/insights-runtime-extractor-7wjhw" May 27 21:30:55.322693 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.322666 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mm292\" (UniqueName: \"kubernetes.io/projected/aafa4c59-c141-4e77-ae60-d6e48d468223-kube-api-access-mm292\") pod \"insights-runtime-extractor-7wjhw\" (UID: \"aafa4c59-c141-4e77-ae60-d6e48d468223\") " pod="openshift-insights/insights-runtime-extractor-7wjhw" May 27 21:30:55.394009 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.393924 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7wjhw" May 27 21:30:55.409895 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.409864 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d78abc3c-065a-43d7-a5e0-3cc662c0d057-ca-trust-extracted\") pod \"image-registry-7c87c4fbbc-psmsw\" (UID: \"d78abc3c-065a-43d7-a5e0-3cc662c0d057\") " pod="openshift-image-registry/image-registry-7c87c4fbbc-psmsw" May 27 21:30:55.410030 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.409900 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d78abc3c-065a-43d7-a5e0-3cc662c0d057-registry-certificates\") pod \"image-registry-7c87c4fbbc-psmsw\" (UID: \"d78abc3c-065a-43d7-a5e0-3cc662c0d057\") " pod="openshift-image-registry/image-registry-7c87c4fbbc-psmsw" May 27 21:30:55.410030 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.409926 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d78abc3c-065a-43d7-a5e0-3cc662c0d057-trusted-ca\") pod \"image-registry-7c87c4fbbc-psmsw\" (UID: \"d78abc3c-065a-43d7-a5e0-3cc662c0d057\") " pod="openshift-image-registry/image-registry-7c87c4fbbc-psmsw" May 27 21:30:55.410030 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.409957 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d78abc3c-065a-43d7-a5e0-3cc662c0d057-registry-tls\") pod \"image-registry-7c87c4fbbc-psmsw\" (UID: \"d78abc3c-065a-43d7-a5e0-3cc662c0d057\") " pod="openshift-image-registry/image-registry-7c87c4fbbc-psmsw" May 27 21:30:55.410030 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.409990 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-n8nms\" (UniqueName: \"kubernetes.io/projected/d78abc3c-065a-43d7-a5e0-3cc662c0d057-kube-api-access-n8nms\") pod \"image-registry-7c87c4fbbc-psmsw\" (UID: \"d78abc3c-065a-43d7-a5e0-3cc662c0d057\") " pod="openshift-image-registry/image-registry-7c87c4fbbc-psmsw" May 27 21:30:55.410226 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.410121 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d78abc3c-065a-43d7-a5e0-3cc662c0d057-image-registry-private-configuration\") pod \"image-registry-7c87c4fbbc-psmsw\" (UID: \"d78abc3c-065a-43d7-a5e0-3cc662c0d057\") " pod="openshift-image-registry/image-registry-7c87c4fbbc-psmsw" May 27 21:30:55.410226 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.410177 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d78abc3c-065a-43d7-a5e0-3cc662c0d057-bound-sa-token\") pod \"image-registry-7c87c4fbbc-psmsw\" (UID: \"d78abc3c-065a-43d7-a5e0-3cc662c0d057\") " pod="openshift-image-registry/image-registry-7c87c4fbbc-psmsw" May 27 21:30:55.410226 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.410223 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d78abc3c-065a-43d7-a5e0-3cc662c0d057-installation-pull-secrets\") pod \"image-registry-7c87c4fbbc-psmsw\" (UID: \"d78abc3c-065a-43d7-a5e0-3cc662c0d057\") " pod="openshift-image-registry/image-registry-7c87c4fbbc-psmsw" May 27 21:30:55.410393 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.410288 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d78abc3c-065a-43d7-a5e0-3cc662c0d057-ca-trust-extracted\") pod \"image-registry-7c87c4fbbc-psmsw\" (UID: \"d78abc3c-065a-43d7-a5e0-3cc662c0d057\") " pod="openshift-image-registry/image-registry-7c87c4fbbc-psmsw" May 27 21:30:55.411009 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.410987 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d78abc3c-065a-43d7-a5e0-3cc662c0d057-registry-certificates\") pod \"image-registry-7c87c4fbbc-psmsw\" (UID: \"d78abc3c-065a-43d7-a5e0-3cc662c0d057\") " pod="openshift-image-registry/image-registry-7c87c4fbbc-psmsw" May 27 21:30:55.411305 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.411283 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d78abc3c-065a-43d7-a5e0-3cc662c0d057-trusted-ca\") pod \"image-registry-7c87c4fbbc-psmsw\" (UID: \"d78abc3c-065a-43d7-a5e0-3cc662c0d057\") " pod="openshift-image-registry/image-registry-7c87c4fbbc-psmsw" May 27 21:30:55.412632 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.412606 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d78abc3c-065a-43d7-a5e0-3cc662c0d057-installation-pull-secrets\") pod \"image-registry-7c87c4fbbc-psmsw\" (UID: \"d78abc3c-065a-43d7-a5e0-3cc662c0d057\") " pod="openshift-image-registry/image-registry-7c87c4fbbc-psmsw" May 27 21:30:55.412776 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.412711 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d78abc3c-065a-43d7-a5e0-3cc662c0d057-registry-tls\") pod \"image-registry-7c87c4fbbc-psmsw\" (UID: \"d78abc3c-065a-43d7-a5e0-3cc662c0d057\") " pod="openshift-image-registry/image-registry-7c87c4fbbc-psmsw" May 27 21:30:55.412776 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.412767 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d78abc3c-065a-43d7-a5e0-3cc662c0d057-image-registry-private-configuration\") pod \"image-registry-7c87c4fbbc-psmsw\" (UID: \"d78abc3c-065a-43d7-a5e0-3cc662c0d057\") " pod="openshift-image-registry/image-registry-7c87c4fbbc-psmsw" May 27 21:30:55.421226 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.421201 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d78abc3c-065a-43d7-a5e0-3cc662c0d057-bound-sa-token\") pod \"image-registry-7c87c4fbbc-psmsw\" (UID: \"d78abc3c-065a-43d7-a5e0-3cc662c0d057\") " pod="openshift-image-registry/image-registry-7c87c4fbbc-psmsw" May 27 21:30:55.425884 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.425859 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8nms\" (UniqueName: \"kubernetes.io/projected/d78abc3c-065a-43d7-a5e0-3cc662c0d057-kube-api-access-n8nms\") pod \"image-registry-7c87c4fbbc-psmsw\" (UID: \"d78abc3c-065a-43d7-a5e0-3cc662c0d057\") " pod="openshift-image-registry/image-registry-7c87c4fbbc-psmsw" May 27 21:30:55.499792 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.499762 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c87c4fbbc-psmsw" May 27 21:30:55.519857 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.519825 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7wjhw"] May 27 21:30:55.522823 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:30:55.522794 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaafa4c59_c141_4e77_ae60_d6e48d468223.slice/crio-c8a37b9826b9d6621d48c408fef8089d739e056591206cadbea811279849d5e2 WatchSource:0}: Error finding container c8a37b9826b9d6621d48c408fef8089d739e056591206cadbea811279849d5e2: Status 404 returned error can't find the container with id c8a37b9826b9d6621d48c408fef8089d739e056591206cadbea811279849d5e2 May 27 21:30:55.582946 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.582916 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7wjhw" event={"ID":"aafa4c59-c141-4e77-ae60-d6e48d468223","Type":"ContainerStarted","Data":"c8a37b9826b9d6621d48c408fef8089d739e056591206cadbea811279849d5e2"} May 27 21:30:55.628233 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.628200 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7c87c4fbbc-psmsw"] May 27 21:30:55.631805 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:30:55.631777 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd78abc3c_065a_43d7_a5e0_3cc662c0d057.slice/crio-6ce7a54d6e55108da17ab8c98f838210db0ed6f515392229b22f72934b4b3d8a WatchSource:0}: Error finding container 6ce7a54d6e55108da17ab8c98f838210db0ed6f515392229b22f72934b4b3d8a: Status 404 returned error can't find the container with id 6ce7a54d6e55108da17ab8c98f838210db0ed6f515392229b22f72934b4b3d8a May 27 21:30:55.813160 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.813117 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7fc93ab3-b601-4fe7-b073-b8702a09404d-metrics-certs\") pod \"network-metrics-daemon-gxt5q\" (UID: \"7fc93ab3-b601-4fe7-b073-b8702a09404d\") " pod="openshift-multus/network-metrics-daemon-gxt5q" May 27 21:30:55.815315 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:55.815289 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7fc93ab3-b601-4fe7-b073-b8702a09404d-metrics-certs\") pod \"network-metrics-daemon-gxt5q\" (UID: \"7fc93ab3-b601-4fe7-b073-b8702a09404d\") " pod="openshift-multus/network-metrics-daemon-gxt5q" May 27 21:30:56.093471 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:56.093445 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-qdzf7\"" May 27 21:30:56.100496 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:56.100480 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gxt5q" May 27 21:30:56.185440 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:56.185378 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-jq28q"] May 27 21:30:56.187669 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:56.187641 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-jq28q" May 27 21:30:56.190154 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:56.190128 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-bxxlx\"" May 27 21:30:56.190304 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:56.190133 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" May 27 21:30:56.198189 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:56.198138 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-jq28q"] May 27 21:30:56.235727 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:56.235693 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-gxt5q"] May 27 21:30:56.293150 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:30:56.293106 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fc93ab3_b601_4fe7_b073_b8702a09404d.slice/crio-6e783bc33414ce8b15c42308c0cd2b24bd5ca6e698bddde9b7326c2506b249a4 WatchSource:0}: Error finding container 6e783bc33414ce8b15c42308c0cd2b24bd5ca6e698bddde9b7326c2506b249a4: Status 404 returned error can't find the container with id 6e783bc33414ce8b15c42308c0cd2b24bd5ca6e698bddde9b7326c2506b249a4 May 27 21:30:56.316951 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:56.316927 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/4da4d84a-f34f-44d3-9312-a8888ac22fa3-tls-certificates\") pod \"prometheus-operator-admission-webhook-6c4bbc67b7-jq28q\" (UID: \"4da4d84a-f34f-44d3-9312-a8888ac22fa3\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-jq28q" May 27 21:30:56.418209 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:56.418179 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/4da4d84a-f34f-44d3-9312-a8888ac22fa3-tls-certificates\") pod \"prometheus-operator-admission-webhook-6c4bbc67b7-jq28q\" (UID: \"4da4d84a-f34f-44d3-9312-a8888ac22fa3\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-jq28q" May 27 21:30:56.418392 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:30:56.418352 2564 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found May 27 21:30:56.418452 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:30:56.418419 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4da4d84a-f34f-44d3-9312-a8888ac22fa3-tls-certificates podName:4da4d84a-f34f-44d3-9312-a8888ac22fa3 nodeName:}" failed. No retries permitted until 2026-05-27 21:30:56.918403835 +0000 UTC m=+130.392608542 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/4da4d84a-f34f-44d3-9312-a8888ac22fa3-tls-certificates") pod "prometheus-operator-admission-webhook-6c4bbc67b7-jq28q" (UID: "4da4d84a-f34f-44d3-9312-a8888ac22fa3") : secret "prometheus-operator-admission-webhook-tls" not found May 27 21:30:56.587186 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:56.587147 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7wjhw" event={"ID":"aafa4c59-c141-4e77-ae60-d6e48d468223","Type":"ContainerStarted","Data":"74f468535cd3e5f1bfddbdaeb43e9c9a13d4d5aafbdf31ae3ad6518f1207d178"} May 27 21:30:56.587186 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:56.587185 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7wjhw" event={"ID":"aafa4c59-c141-4e77-ae60-d6e48d468223","Type":"ContainerStarted","Data":"ca85627365ac6d24ea7049e8eae4df1c9aaf037c8a760b71e07c832cea187df9"} May 27 21:30:56.588444 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:56.588418 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7c87c4fbbc-psmsw" event={"ID":"d78abc3c-065a-43d7-a5e0-3cc662c0d057","Type":"ContainerStarted","Data":"32af2a8dff9bee9c2b8a1acc903719ce048ed1b3a5ee276a17c84bf63152b2b6"} May 27 21:30:56.588526 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:56.588451 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7c87c4fbbc-psmsw" event={"ID":"d78abc3c-065a-43d7-a5e0-3cc662c0d057","Type":"ContainerStarted","Data":"6ce7a54d6e55108da17ab8c98f838210db0ed6f515392229b22f72934b4b3d8a"} May 27 21:30:56.588526 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:56.588485 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7c87c4fbbc-psmsw" May 27 21:30:56.589548 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:56.589526 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gxt5q" event={"ID":"7fc93ab3-b601-4fe7-b073-b8702a09404d","Type":"ContainerStarted","Data":"6e783bc33414ce8b15c42308c0cd2b24bd5ca6e698bddde9b7326c2506b249a4"} May 27 21:30:56.606897 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:56.606803 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7c87c4fbbc-psmsw" podStartSLOduration=1.606789746 podStartE2EDuration="1.606789746s" podCreationTimestamp="2026-05-27 21:30:55 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-27 21:30:56.606721273 +0000 UTC m=+130.080926006" watchObservedRunningTime="2026-05-27 21:30:56.606789746 +0000 UTC m=+130.080994476" May 27 21:30:56.923893 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:56.923846 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/4da4d84a-f34f-44d3-9312-a8888ac22fa3-tls-certificates\") pod \"prometheus-operator-admission-webhook-6c4bbc67b7-jq28q\" (UID: \"4da4d84a-f34f-44d3-9312-a8888ac22fa3\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-jq28q" May 27 21:30:56.926418 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:56.926387 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/4da4d84a-f34f-44d3-9312-a8888ac22fa3-tls-certificates\") pod \"prometheus-operator-admission-webhook-6c4bbc67b7-jq28q\" (UID: \"4da4d84a-f34f-44d3-9312-a8888ac22fa3\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-jq28q" May 27 21:30:57.104602 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:57.104560 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-jq28q" May 27 21:30:57.257141 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:57.257107 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-jq28q"] May 27 21:30:57.467923 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:30:57.467848 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4da4d84a_f34f_44d3_9312_a8888ac22fa3.slice/crio-46516e5f71645b6085e1a958b0e093711930dd3db45257626d74049167618e89 WatchSource:0}: Error finding container 46516e5f71645b6085e1a958b0e093711930dd3db45257626d74049167618e89: Status 404 returned error can't find the container with id 46516e5f71645b6085e1a958b0e093711930dd3db45257626d74049167618e89 May 27 21:30:57.594332 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:57.594299 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gxt5q" event={"ID":"7fc93ab3-b601-4fe7-b073-b8702a09404d","Type":"ContainerStarted","Data":"4c58d3fba6758b9b69022047fe7bdd098a09662b0471e45e0ccefe9191f7ddaa"} May 27 21:30:57.595613 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:57.595579 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-jq28q" event={"ID":"4da4d84a-f34f-44d3-9312-a8888ac22fa3","Type":"ContainerStarted","Data":"46516e5f71645b6085e1a958b0e093711930dd3db45257626d74049167618e89"} May 27 21:30:58.602845 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:58.602804 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7wjhw" event={"ID":"aafa4c59-c141-4e77-ae60-d6e48d468223","Type":"ContainerStarted","Data":"35575dfa2be1899ddc2df80a2efe7e07eaa2ddd5edda84997cd09b43aa5b7c59"} May 27 21:30:58.604655 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:58.604618 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gxt5q" event={"ID":"7fc93ab3-b601-4fe7-b073-b8702a09404d","Type":"ContainerStarted","Data":"495c42ab940c038a29e6512146c029a666d8a192c87a7d1bd6510b84f2d8809d"} May 27 21:30:58.624299 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:58.624228 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-7wjhw" podStartSLOduration=1.152906834 podStartE2EDuration="3.624213471s" podCreationTimestamp="2026-05-27 21:30:55 +0000 UTC" firstStartedPulling="2026-05-27 21:30:55.583162481 +0000 UTC m=+129.057367196" lastFinishedPulling="2026-05-27 21:30:58.05446911 +0000 UTC m=+131.528673833" observedRunningTime="2026-05-27 21:30:58.62336817 +0000 UTC m=+132.097572891" watchObservedRunningTime="2026-05-27 21:30:58.624213471 +0000 UTC m=+132.098418201" May 27 21:30:58.648388 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:58.648330 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-gxt5q" podStartSLOduration=130.44782597 podStartE2EDuration="2m11.648311568s" podCreationTimestamp="2026-05-27 21:28:47 +0000 UTC" firstStartedPulling="2026-05-27 21:30:56.295213198 +0000 UTC m=+129.769417921" lastFinishedPulling="2026-05-27 21:30:57.495698804 +0000 UTC m=+130.969903519" observedRunningTime="2026-05-27 21:30:58.647298987 +0000 UTC m=+132.121503721" watchObservedRunningTime="2026-05-27 21:30:58.648311568 +0000 UTC m=+132.122516298" May 27 21:30:59.608357 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:59.608311 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-jq28q" event={"ID":"4da4d84a-f34f-44d3-9312-a8888ac22fa3","Type":"ContainerStarted","Data":"f5f6556fb9737ea9738088c09fa2fc0e0eda4944225c7699ce4fde338f5de3ce"} May 27 21:30:59.639693 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:30:59.639637 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-jq28q" podStartSLOduration=2.237084237 podStartE2EDuration="3.639619667s" podCreationTimestamp="2026-05-27 21:30:56 +0000 UTC" firstStartedPulling="2026-05-27 21:30:57.470191117 +0000 UTC m=+130.944395827" lastFinishedPulling="2026-05-27 21:30:58.872726545 +0000 UTC m=+132.346931257" observedRunningTime="2026-05-27 21:30:59.639412689 +0000 UTC m=+133.113617420" watchObservedRunningTime="2026-05-27 21:30:59.639619667 +0000 UTC m=+133.113824438" May 27 21:31:00.611126 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:00.611084 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-jq28q" May 27 21:31:00.616786 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:00.616757 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-jq28q" May 27 21:31:01.260177 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:01.260144 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7779bbdd78-jhdrv"] May 27 21:31:01.263181 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:01.263163 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7779bbdd78-jhdrv" May 27 21:31:01.266240 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:01.266218 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" May 27 21:31:01.267342 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:01.267319 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" May 27 21:31:01.267517 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:01.267490 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" May 27 21:31:01.267637 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:01.267519 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" May 27 21:31:01.267637 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:01.267553 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" May 27 21:31:01.267637 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:01.267583 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-qgxht\"" May 27 21:31:01.275454 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:01.275427 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7779bbdd78-jhdrv"] May 27 21:31:01.359438 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:01.359405 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlqzq\" (UniqueName: \"kubernetes.io/projected/ece3e9ae-4d45-45d6-a435-df9cb81ebb42-kube-api-access-nlqzq\") pod \"prometheus-operator-7779bbdd78-jhdrv\" (UID: \"ece3e9ae-4d45-45d6-a435-df9cb81ebb42\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-jhdrv" May 27 21:31:01.359438 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:01.359444 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ece3e9ae-4d45-45d6-a435-df9cb81ebb42-metrics-client-ca\") pod \"prometheus-operator-7779bbdd78-jhdrv\" (UID: \"ece3e9ae-4d45-45d6-a435-df9cb81ebb42\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-jhdrv" May 27 21:31:01.359652 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:01.359473 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ece3e9ae-4d45-45d6-a435-df9cb81ebb42-prometheus-operator-tls\") pod \"prometheus-operator-7779bbdd78-jhdrv\" (UID: \"ece3e9ae-4d45-45d6-a435-df9cb81ebb42\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-jhdrv" May 27 21:31:01.359652 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:01.359555 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ece3e9ae-4d45-45d6-a435-df9cb81ebb42-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7779bbdd78-jhdrv\" (UID: \"ece3e9ae-4d45-45d6-a435-df9cb81ebb42\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-jhdrv" May 27 21:31:01.460394 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:01.460341 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nlqzq\" (UniqueName: \"kubernetes.io/projected/ece3e9ae-4d45-45d6-a435-df9cb81ebb42-kube-api-access-nlqzq\") pod \"prometheus-operator-7779bbdd78-jhdrv\" (UID: \"ece3e9ae-4d45-45d6-a435-df9cb81ebb42\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-jhdrv" May 27 21:31:01.460394 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:01.460401 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ece3e9ae-4d45-45d6-a435-df9cb81ebb42-metrics-client-ca\") pod \"prometheus-operator-7779bbdd78-jhdrv\" (UID: \"ece3e9ae-4d45-45d6-a435-df9cb81ebb42\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-jhdrv" May 27 21:31:01.460555 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:01.460444 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ece3e9ae-4d45-45d6-a435-df9cb81ebb42-prometheus-operator-tls\") pod \"prometheus-operator-7779bbdd78-jhdrv\" (UID: \"ece3e9ae-4d45-45d6-a435-df9cb81ebb42\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-jhdrv" May 27 21:31:01.460555 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:01.460473 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ece3e9ae-4d45-45d6-a435-df9cb81ebb42-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7779bbdd78-jhdrv\" (UID: \"ece3e9ae-4d45-45d6-a435-df9cb81ebb42\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-jhdrv" May 27 21:31:01.460982 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:01.460962 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ece3e9ae-4d45-45d6-a435-df9cb81ebb42-metrics-client-ca\") pod \"prometheus-operator-7779bbdd78-jhdrv\" (UID: \"ece3e9ae-4d45-45d6-a435-df9cb81ebb42\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-jhdrv" May 27 21:31:01.462959 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:01.462937 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ece3e9ae-4d45-45d6-a435-df9cb81ebb42-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7779bbdd78-jhdrv\" (UID: \"ece3e9ae-4d45-45d6-a435-df9cb81ebb42\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-jhdrv" May 27 21:31:01.463034 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:01.463013 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ece3e9ae-4d45-45d6-a435-df9cb81ebb42-prometheus-operator-tls\") pod \"prometheus-operator-7779bbdd78-jhdrv\" (UID: \"ece3e9ae-4d45-45d6-a435-df9cb81ebb42\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-jhdrv" May 27 21:31:01.469648 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:01.469614 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlqzq\" (UniqueName: \"kubernetes.io/projected/ece3e9ae-4d45-45d6-a435-df9cb81ebb42-kube-api-access-nlqzq\") pod \"prometheus-operator-7779bbdd78-jhdrv\" (UID: \"ece3e9ae-4d45-45d6-a435-df9cb81ebb42\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-jhdrv" May 27 21:31:01.571878 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:01.571792 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7779bbdd78-jhdrv" May 27 21:31:01.696441 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:01.696407 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7779bbdd78-jhdrv"] May 27 21:31:01.699754 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:31:01.699725 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podece3e9ae_4d45_45d6_a435_df9cb81ebb42.slice/crio-45849ee62452c5de231db89a9ae1e29a4cc7dde0de5ca0d2e3889fd471fd9b69 WatchSource:0}: Error finding container 45849ee62452c5de231db89a9ae1e29a4cc7dde0de5ca0d2e3889fd471fd9b69: Status 404 returned error can't find the container with id 45849ee62452c5de231db89a9ae1e29a4cc7dde0de5ca0d2e3889fd471fd9b69 May 27 21:31:02.079748 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:02.079715 2564 scope.go:117] "RemoveContainer" containerID="62ba8701691a698f432ece42c92493f5e343140dadc33cb2f8bf9afcf7556e23" May 27 21:31:02.079923 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:31:02.079889 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-76dff48d59-d9gcd_openshift-console-operator(16b8cf1e-1181-43fc-92e0-635b8523248d)\"" pod="openshift-console-operator/console-operator-76dff48d59-d9gcd" podUID="16b8cf1e-1181-43fc-92e0-635b8523248d" May 27 21:31:02.620563 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:02.620523 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7779bbdd78-jhdrv" event={"ID":"ece3e9ae-4d45-45d6-a435-df9cb81ebb42","Type":"ContainerStarted","Data":"45849ee62452c5de231db89a9ae1e29a4cc7dde0de5ca0d2e3889fd471fd9b69"} May 27 21:31:03.625389 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:03.625349 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7779bbdd78-jhdrv" event={"ID":"ece3e9ae-4d45-45d6-a435-df9cb81ebb42","Type":"ContainerStarted","Data":"becd5a86bc5d2123b5b7daa8c3fd1e13f95d9b2687ff4207806337d424229f41"} May 27 21:31:03.625389 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:03.625389 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7779bbdd78-jhdrv" event={"ID":"ece3e9ae-4d45-45d6-a435-df9cb81ebb42","Type":"ContainerStarted","Data":"4e403bea39af0c74b9de76cc096c90338bcc84ca1b881eb29c69ea37e614ff43"} May 27 21:31:03.644628 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:03.644578 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7779bbdd78-jhdrv" podStartSLOduration=1.224571553 podStartE2EDuration="2.644563535s" podCreationTimestamp="2026-05-27 21:31:01 +0000 UTC" firstStartedPulling="2026-05-27 21:31:01.701484058 +0000 UTC m=+135.175688766" lastFinishedPulling="2026-05-27 21:31:03.121476037 +0000 UTC m=+136.595680748" observedRunningTime="2026-05-27 21:31:03.643420309 +0000 UTC m=+137.117625038" watchObservedRunningTime="2026-05-27 21:31:03.644563535 +0000 UTC m=+137.118768272" May 27 21:31:05.598712 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:05.598671 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-54fd6bc5c4-77nkf"] May 27 21:31:05.601435 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:05.601414 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-77nkf" May 27 21:31:05.603948 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:05.603920 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" May 27 21:31:05.604381 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:05.604365 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" May 27 21:31:05.604976 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:05.604959 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" May 27 21:31:05.605052 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:05.604985 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-qhswp\"" May 27 21:31:05.612233 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:05.612204 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-54fd6bc5c4-77nkf"] May 27 21:31:05.621556 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:05.621531 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-8prph"] May 27 21:31:05.623664 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:05.623648 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8prph" May 27 21:31:05.628298 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:05.628260 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-2qckr\"" May 27 21:31:05.628298 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:05.628285 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" May 27 21:31:05.628495 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:05.628473 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" May 27 21:31:05.628786 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:05.628771 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" May 27 21:31:05.795576 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:05.795542 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/030b9490-f0b8-4a41-90a9-da20af4103be-kube-state-metrics-tls\") pod \"kube-state-metrics-54fd6bc5c4-77nkf\" (UID: \"030b9490-f0b8-4a41-90a9-da20af4103be\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-77nkf" May 27 21:31:05.795576 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:05.795585 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/32f3075a-ac99-468a-a2b9-c76f16897682-sys\") pod \"node-exporter-8prph\" (UID: \"32f3075a-ac99-468a-a2b9-c76f16897682\") " pod="openshift-monitoring/node-exporter-8prph" May 27 21:31:05.795816 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:05.795607 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/32f3075a-ac99-468a-a2b9-c76f16897682-node-exporter-tls\") pod \"node-exporter-8prph\" (UID: \"32f3075a-ac99-468a-a2b9-c76f16897682\") " pod="openshift-monitoring/node-exporter-8prph" May 27 21:31:05.795816 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:05.795679 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/030b9490-f0b8-4a41-90a9-da20af4103be-volume-directive-shadow\") pod \"kube-state-metrics-54fd6bc5c4-77nkf\" (UID: \"030b9490-f0b8-4a41-90a9-da20af4103be\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-77nkf" May 27 21:31:05.795816 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:05.795733 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/32f3075a-ac99-468a-a2b9-c76f16897682-node-exporter-textfile\") pod \"node-exporter-8prph\" (UID: \"32f3075a-ac99-468a-a2b9-c76f16897682\") " pod="openshift-monitoring/node-exporter-8prph" May 27 21:31:05.795816 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:05.795753 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/030b9490-f0b8-4a41-90a9-da20af4103be-metrics-client-ca\") pod \"kube-state-metrics-54fd6bc5c4-77nkf\" (UID: \"030b9490-f0b8-4a41-90a9-da20af4103be\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-77nkf" May 27 21:31:05.795816 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:05.795775 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/32f3075a-ac99-468a-a2b9-c76f16897682-node-exporter-accelerators-collector-config\") pod \"node-exporter-8prph\" (UID: \"32f3075a-ac99-468a-a2b9-c76f16897682\") " pod="openshift-monitoring/node-exporter-8prph" May 27 21:31:05.795816 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:05.795796 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/32f3075a-ac99-468a-a2b9-c76f16897682-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8prph\" (UID: \"32f3075a-ac99-468a-a2b9-c76f16897682\") " pod="openshift-monitoring/node-exporter-8prph" May 27 21:31:05.796021 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:05.795874 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n48h7\" (UniqueName: \"kubernetes.io/projected/32f3075a-ac99-468a-a2b9-c76f16897682-kube-api-access-n48h7\") pod \"node-exporter-8prph\" (UID: \"32f3075a-ac99-468a-a2b9-c76f16897682\") " pod="openshift-monitoring/node-exporter-8prph" May 27 21:31:05.796021 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:05.795979 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/32f3075a-ac99-468a-a2b9-c76f16897682-metrics-client-ca\") pod \"node-exporter-8prph\" (UID: \"32f3075a-ac99-468a-a2b9-c76f16897682\") " pod="openshift-monitoring/node-exporter-8prph" May 27 21:31:05.796021 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:05.796013 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dl2pz\" (UniqueName: \"kubernetes.io/projected/030b9490-f0b8-4a41-90a9-da20af4103be-kube-api-access-dl2pz\") pod \"kube-state-metrics-54fd6bc5c4-77nkf\" (UID: \"030b9490-f0b8-4a41-90a9-da20af4103be\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-77nkf" May 27 21:31:05.796135 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:05.796035 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/32f3075a-ac99-468a-a2b9-c76f16897682-root\") pod \"node-exporter-8prph\" (UID: \"32f3075a-ac99-468a-a2b9-c76f16897682\") " pod="openshift-monitoring/node-exporter-8prph" May 27 21:31:05.796135 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:05.796081 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/030b9490-f0b8-4a41-90a9-da20af4103be-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-54fd6bc5c4-77nkf\" (UID: \"030b9490-f0b8-4a41-90a9-da20af4103be\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-77nkf" May 27 21:31:05.796135 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:05.796108 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/030b9490-f0b8-4a41-90a9-da20af4103be-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-54fd6bc5c4-77nkf\" (UID: \"030b9490-f0b8-4a41-90a9-da20af4103be\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-77nkf" May 27 21:31:05.796232 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:05.796150 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/32f3075a-ac99-468a-a2b9-c76f16897682-node-exporter-wtmp\") pod \"node-exporter-8prph\" (UID: \"32f3075a-ac99-468a-a2b9-c76f16897682\") " pod="openshift-monitoring/node-exporter-8prph" May 27 21:31:05.896857 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:05.896772 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dl2pz\" (UniqueName: \"kubernetes.io/projected/030b9490-f0b8-4a41-90a9-da20af4103be-kube-api-access-dl2pz\") pod \"kube-state-metrics-54fd6bc5c4-77nkf\" (UID: \"030b9490-f0b8-4a41-90a9-da20af4103be\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-77nkf" May 27 21:31:05.896857 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:05.896807 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/32f3075a-ac99-468a-a2b9-c76f16897682-root\") pod \"node-exporter-8prph\" (UID: \"32f3075a-ac99-468a-a2b9-c76f16897682\") " pod="openshift-monitoring/node-exporter-8prph" May 27 21:31:05.896857 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:05.896843 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/030b9490-f0b8-4a41-90a9-da20af4103be-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-54fd6bc5c4-77nkf\" (UID: \"030b9490-f0b8-4a41-90a9-da20af4103be\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-77nkf" May 27 21:31:05.897117 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:05.896870 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/030b9490-f0b8-4a41-90a9-da20af4103be-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-54fd6bc5c4-77nkf\" (UID: \"030b9490-f0b8-4a41-90a9-da20af4103be\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-77nkf" May 27 21:31:05.897117 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:05.896895 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/32f3075a-ac99-468a-a2b9-c76f16897682-root\") pod \"node-exporter-8prph\" (UID: \"32f3075a-ac99-468a-a2b9-c76f16897682\") " pod="openshift-monitoring/node-exporter-8prph" May 27 21:31:05.897117 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:05.896904 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/32f3075a-ac99-468a-a2b9-c76f16897682-node-exporter-wtmp\") pod \"node-exporter-8prph\" (UID: \"32f3075a-ac99-468a-a2b9-c76f16897682\") " pod="openshift-monitoring/node-exporter-8prph" May 27 21:31:05.897117 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:05.896980 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/030b9490-f0b8-4a41-90a9-da20af4103be-kube-state-metrics-tls\") pod \"kube-state-metrics-54fd6bc5c4-77nkf\" (UID: \"030b9490-f0b8-4a41-90a9-da20af4103be\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-77nkf" May 27 21:31:05.897117 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:05.897013 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/32f3075a-ac99-468a-a2b9-c76f16897682-sys\") pod \"node-exporter-8prph\" (UID: \"32f3075a-ac99-468a-a2b9-c76f16897682\") " pod="openshift-monitoring/node-exporter-8prph" May 27 21:31:05.897117 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:05.897035 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/32f3075a-ac99-468a-a2b9-c76f16897682-node-exporter-tls\") pod \"node-exporter-8prph\" (UID: \"32f3075a-ac99-468a-a2b9-c76f16897682\") " pod="openshift-monitoring/node-exporter-8prph" May 27 21:31:05.897117 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:05.897038 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/32f3075a-ac99-468a-a2b9-c76f16897682-node-exporter-wtmp\") pod \"node-exporter-8prph\" (UID: \"32f3075a-ac99-468a-a2b9-c76f16897682\") " pod="openshift-monitoring/node-exporter-8prph" May 27 21:31:05.897117 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:05.897061 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/030b9490-f0b8-4a41-90a9-da20af4103be-volume-directive-shadow\") pod \"kube-state-metrics-54fd6bc5c4-77nkf\" (UID: \"030b9490-f0b8-4a41-90a9-da20af4103be\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-77nkf" May 27 21:31:05.897117 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:05.897119 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/32f3075a-ac99-468a-a2b9-c76f16897682-node-exporter-textfile\") pod \"node-exporter-8prph\" (UID: \"32f3075a-ac99-468a-a2b9-c76f16897682\") " pod="openshift-monitoring/node-exporter-8prph" May 27 21:31:05.897569 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:05.897163 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/030b9490-f0b8-4a41-90a9-da20af4103be-metrics-client-ca\") pod \"kube-state-metrics-54fd6bc5c4-77nkf\" (UID: \"030b9490-f0b8-4a41-90a9-da20af4103be\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-77nkf" May 27 21:31:05.897569 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:05.897196 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/32f3075a-ac99-468a-a2b9-c76f16897682-node-exporter-accelerators-collector-config\") pod \"node-exporter-8prph\" (UID: \"32f3075a-ac99-468a-a2b9-c76f16897682\") " pod="openshift-monitoring/node-exporter-8prph" May 27 21:31:05.897569 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:05.897313 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/32f3075a-ac99-468a-a2b9-c76f16897682-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8prph\" (UID: \"32f3075a-ac99-468a-a2b9-c76f16897682\") " pod="openshift-monitoring/node-exporter-8prph" May 27 21:31:05.897569 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:05.897359 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-n48h7\" (UniqueName: \"kubernetes.io/projected/32f3075a-ac99-468a-a2b9-c76f16897682-kube-api-access-n48h7\") pod \"node-exporter-8prph\" (UID: \"32f3075a-ac99-468a-a2b9-c76f16897682\") " pod="openshift-monitoring/node-exporter-8prph" May 27 21:31:05.897569 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:05.897427 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/32f3075a-ac99-468a-a2b9-c76f16897682-metrics-client-ca\") pod \"node-exporter-8prph\" (UID: \"32f3075a-ac99-468a-a2b9-c76f16897682\") " pod="openshift-monitoring/node-exporter-8prph" May 27 21:31:05.897569 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:05.897466 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/030b9490-f0b8-4a41-90a9-da20af4103be-volume-directive-shadow\") pod \"kube-state-metrics-54fd6bc5c4-77nkf\" (UID: \"030b9490-f0b8-4a41-90a9-da20af4103be\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-77nkf" May 27 21:31:05.897866 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:05.897749 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/030b9490-f0b8-4a41-90a9-da20af4103be-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-54fd6bc5c4-77nkf\" (UID: \"030b9490-f0b8-4a41-90a9-da20af4103be\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-77nkf" May 27 21:31:05.897866 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:05.897755 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/32f3075a-ac99-468a-a2b9-c76f16897682-node-exporter-textfile\") pod \"node-exporter-8prph\" (UID: \"32f3075a-ac99-468a-a2b9-c76f16897682\") " pod="openshift-monitoring/node-exporter-8prph" May 27 21:31:05.898178 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:05.898052 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/32f3075a-ac99-468a-a2b9-c76f16897682-sys\") pod \"node-exporter-8prph\" (UID: \"32f3075a-ac99-468a-a2b9-c76f16897682\") " pod="openshift-monitoring/node-exporter-8prph" May 27 21:31:05.898370 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:05.898287 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/32f3075a-ac99-468a-a2b9-c76f16897682-metrics-client-ca\") pod \"node-exporter-8prph\" (UID: \"32f3075a-ac99-468a-a2b9-c76f16897682\") " pod="openshift-monitoring/node-exporter-8prph" May 27 21:31:05.898370 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:05.898346 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/32f3075a-ac99-468a-a2b9-c76f16897682-node-exporter-accelerators-collector-config\") pod \"node-exporter-8prph\" (UID: \"32f3075a-ac99-468a-a2b9-c76f16897682\") " pod="openshift-monitoring/node-exporter-8prph" May 27 21:31:05.898544 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:05.898453 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/030b9490-f0b8-4a41-90a9-da20af4103be-metrics-client-ca\") pod \"kube-state-metrics-54fd6bc5c4-77nkf\" (UID: \"030b9490-f0b8-4a41-90a9-da20af4103be\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-77nkf" May 27 21:31:05.900030 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:05.900009 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/030b9490-f0b8-4a41-90a9-da20af4103be-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-54fd6bc5c4-77nkf\" (UID: \"030b9490-f0b8-4a41-90a9-da20af4103be\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-77nkf" May 27 21:31:05.900126 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:05.900014 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/32f3075a-ac99-468a-a2b9-c76f16897682-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8prph\" (UID: \"32f3075a-ac99-468a-a2b9-c76f16897682\") " pod="openshift-monitoring/node-exporter-8prph" May 27 21:31:05.900240 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:05.900220 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/030b9490-f0b8-4a41-90a9-da20af4103be-kube-state-metrics-tls\") pod \"kube-state-metrics-54fd6bc5c4-77nkf\" (UID: \"030b9490-f0b8-4a41-90a9-da20af4103be\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-77nkf" May 27 21:31:05.901256 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:05.901237 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/32f3075a-ac99-468a-a2b9-c76f16897682-node-exporter-tls\") pod \"node-exporter-8prph\" (UID: \"32f3075a-ac99-468a-a2b9-c76f16897682\") " pod="openshift-monitoring/node-exporter-8prph" May 27 21:31:05.908869 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:05.908848 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n48h7\" (UniqueName: \"kubernetes.io/projected/32f3075a-ac99-468a-a2b9-c76f16897682-kube-api-access-n48h7\") pod \"node-exporter-8prph\" (UID: \"32f3075a-ac99-468a-a2b9-c76f16897682\") " pod="openshift-monitoring/node-exporter-8prph" May 27 21:31:05.909222 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:05.909203 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dl2pz\" (UniqueName: \"kubernetes.io/projected/030b9490-f0b8-4a41-90a9-da20af4103be-kube-api-access-dl2pz\") pod \"kube-state-metrics-54fd6bc5c4-77nkf\" (UID: \"030b9490-f0b8-4a41-90a9-da20af4103be\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-77nkf" May 27 21:31:05.912890 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:05.912871 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-77nkf" May 27 21:31:05.935909 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:05.935883 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8prph" May 27 21:31:05.945408 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:31:05.945375 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32f3075a_ac99_468a_a2b9_c76f16897682.slice/crio-0477893331bbdb0e0141df549af1230e51da153875aec0b8a8bceee584318aaf WatchSource:0}: Error finding container 0477893331bbdb0e0141df549af1230e51da153875aec0b8a8bceee584318aaf: Status 404 returned error can't find the container with id 0477893331bbdb0e0141df549af1230e51da153875aec0b8a8bceee584318aaf May 27 21:31:06.062407 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:06.062244 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-54fd6bc5c4-77nkf"] May 27 21:31:06.065958 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:31:06.065929 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod030b9490_f0b8_4a41_90a9_da20af4103be.slice/crio-b8a531cc4d2768de9fe29f78169adce1c782f7655a364a85373c5ff2eb08c83a WatchSource:0}: Error finding container b8a531cc4d2768de9fe29f78169adce1c782f7655a364a85373c5ff2eb08c83a: Status 404 returned error can't find the container with id b8a531cc4d2768de9fe29f78169adce1c782f7655a364a85373c5ff2eb08c83a May 27 21:31:06.636998 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:06.636957 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-77nkf" event={"ID":"030b9490-f0b8-4a41-90a9-da20af4103be","Type":"ContainerStarted","Data":"b8a531cc4d2768de9fe29f78169adce1c782f7655a364a85373c5ff2eb08c83a"} May 27 21:31:06.638085 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:06.638060 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8prph" event={"ID":"32f3075a-ac99-468a-a2b9-c76f16897682","Type":"ContainerStarted","Data":"0477893331bbdb0e0141df549af1230e51da153875aec0b8a8bceee584318aaf"} May 27 21:31:07.642802 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:07.642766 2564 generic.go:358] "Generic (PLEG): container finished" podID="32f3075a-ac99-468a-a2b9-c76f16897682" containerID="b60df280486e827962eaa3b9829d5814a1955f676d1c554ac75461bbbe5a3e19" exitCode=0 May 27 21:31:07.643200 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:07.642852 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8prph" event={"ID":"32f3075a-ac99-468a-a2b9-c76f16897682","Type":"ContainerDied","Data":"b60df280486e827962eaa3b9829d5814a1955f676d1c554ac75461bbbe5a3e19"} May 27 21:31:07.644492 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:07.644470 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-77nkf" event={"ID":"030b9490-f0b8-4a41-90a9-da20af4103be","Type":"ContainerStarted","Data":"6135836cd874be24f21a8456902cc9d3d3c23558a4d0993ebdee481fc59c0252"} May 27 21:31:08.649640 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:08.649600 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-77nkf" event={"ID":"030b9490-f0b8-4a41-90a9-da20af4103be","Type":"ContainerStarted","Data":"197f391678e274190f1fc92447bb9dfa681883392575c88ebcac3d3d17fd0ec8"} May 27 21:31:08.649640 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:08.649644 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-77nkf" event={"ID":"030b9490-f0b8-4a41-90a9-da20af4103be","Type":"ContainerStarted","Data":"2e82f71c58c760fbec89403fb3cb956539f7f4f4a55791b1f16b99f99b7bfded"} May 27 21:31:08.651665 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:08.651636 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8prph" event={"ID":"32f3075a-ac99-468a-a2b9-c76f16897682","Type":"ContainerStarted","Data":"f7904729d994cb907e6326570a0448470b012e3d2ab0429a6dad124e1a2113fd"} May 27 21:31:08.651782 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:08.651669 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8prph" event={"ID":"32f3075a-ac99-468a-a2b9-c76f16897682","Type":"ContainerStarted","Data":"38177cd2d96b6b290f5bcb0faf3cfbcc3daee925aefa812623c1fadc77d89336"} May 27 21:31:08.673855 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:08.673802 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-77nkf" podStartSLOduration=2.19107211 podStartE2EDuration="3.673783997s" podCreationTimestamp="2026-05-27 21:31:05 +0000 UTC" firstStartedPulling="2026-05-27 21:31:06.06803919 +0000 UTC m=+139.542243901" lastFinishedPulling="2026-05-27 21:31:07.550751066 +0000 UTC m=+141.024955788" observedRunningTime="2026-05-27 21:31:08.67350095 +0000 UTC m=+142.147705681" watchObservedRunningTime="2026-05-27 21:31:08.673783997 +0000 UTC m=+142.147988728" May 27 21:31:08.706676 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:08.706625 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-8prph" podStartSLOduration=2.475850314 podStartE2EDuration="3.706611467s" podCreationTimestamp="2026-05-27 21:31:05 +0000 UTC" firstStartedPulling="2026-05-27 21:31:05.947310205 +0000 UTC m=+139.421514913" lastFinishedPulling="2026-05-27 21:31:07.178071358 +0000 UTC m=+140.652276066" observedRunningTime="2026-05-27 21:31:08.70561703 +0000 UTC m=+142.179821761" watchObservedRunningTime="2026-05-27 21:31:08.706611467 +0000 UTC m=+142.180816197" May 27 21:31:10.430424 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:10.430382 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-665c9c8584-xshg4"] May 27 21:31:10.432878 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:10.432856 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-665c9c8584-xshg4" May 27 21:31:10.435555 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:10.435528 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"default-dockercfg-zr4m8\"" May 27 21:31:10.435932 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:10.435913 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" May 27 21:31:10.447646 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:10.447623 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-665c9c8584-xshg4"] May 27 21:31:10.539433 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:10.539400 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b7a8fe62-9758-4449-b720-78d9c0842165-monitoring-plugin-cert\") pod \"monitoring-plugin-665c9c8584-xshg4\" (UID: \"b7a8fe62-9758-4449-b720-78d9c0842165\") " pod="openshift-monitoring/monitoring-plugin-665c9c8584-xshg4" May 27 21:31:10.640796 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:10.640765 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b7a8fe62-9758-4449-b720-78d9c0842165-monitoring-plugin-cert\") pod \"monitoring-plugin-665c9c8584-xshg4\" (UID: \"b7a8fe62-9758-4449-b720-78d9c0842165\") " pod="openshift-monitoring/monitoring-plugin-665c9c8584-xshg4" May 27 21:31:10.640926 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:31:10.640913 2564 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found May 27 21:31:10.640997 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:31:10.640985 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7a8fe62-9758-4449-b720-78d9c0842165-monitoring-plugin-cert podName:b7a8fe62-9758-4449-b720-78d9c0842165 nodeName:}" failed. No retries permitted until 2026-05-27 21:31:11.14096939 +0000 UTC m=+144.615174103 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/b7a8fe62-9758-4449-b720-78d9c0842165-monitoring-plugin-cert") pod "monitoring-plugin-665c9c8584-xshg4" (UID: "b7a8fe62-9758-4449-b720-78d9c0842165") : secret "monitoring-plugin-cert" not found May 27 21:31:11.144951 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:11.144918 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b7a8fe62-9758-4449-b720-78d9c0842165-monitoring-plugin-cert\") pod \"monitoring-plugin-665c9c8584-xshg4\" (UID: \"b7a8fe62-9758-4449-b720-78d9c0842165\") " pod="openshift-monitoring/monitoring-plugin-665c9c8584-xshg4" May 27 21:31:11.147318 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:11.147287 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b7a8fe62-9758-4449-b720-78d9c0842165-monitoring-plugin-cert\") pod \"monitoring-plugin-665c9c8584-xshg4\" (UID: \"b7a8fe62-9758-4449-b720-78d9c0842165\") " pod="openshift-monitoring/monitoring-plugin-665c9c8584-xshg4" May 27 21:31:11.343180 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:11.343146 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-665c9c8584-xshg4" May 27 21:31:11.472434 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:11.472408 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-665c9c8584-xshg4"] May 27 21:31:11.475033 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:31:11.475006 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7a8fe62_9758_4449_b720_78d9c0842165.slice/crio-625259a1e05659b8629c8db32a9e22717d568ff2702543c9e50b85fa3829624d WatchSource:0}: Error finding container 625259a1e05659b8629c8db32a9e22717d568ff2702543c9e50b85fa3829624d: Status 404 returned error can't find the container with id 625259a1e05659b8629c8db32a9e22717d568ff2702543c9e50b85fa3829624d May 27 21:31:11.661049 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:11.661016 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-665c9c8584-xshg4" event={"ID":"b7a8fe62-9758-4449-b720-78d9c0842165","Type":"ContainerStarted","Data":"625259a1e05659b8629c8db32a9e22717d568ff2702543c9e50b85fa3829624d"} May 27 21:31:13.669407 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:13.669374 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-665c9c8584-xshg4" event={"ID":"b7a8fe62-9758-4449-b720-78d9c0842165","Type":"ContainerStarted","Data":"70f73bcffd4f307726e9e6e836a872f335f99ad6839a52aa12f53730f7dbd25e"} May 27 21:31:13.669797 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:13.669585 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-665c9c8584-xshg4" May 27 21:31:13.674722 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:13.674700 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-665c9c8584-xshg4" May 27 21:31:13.694452 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:13.694410 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-665c9c8584-xshg4" podStartSLOduration=2.36790217 podStartE2EDuration="3.694397294s" podCreationTimestamp="2026-05-27 21:31:10 +0000 UTC" firstStartedPulling="2026-05-27 21:31:11.476693877 +0000 UTC m=+144.950898589" lastFinishedPulling="2026-05-27 21:31:12.803188992 +0000 UTC m=+146.277393713" observedRunningTime="2026-05-27 21:31:13.693631318 +0000 UTC m=+147.167836047" watchObservedRunningTime="2026-05-27 21:31:13.694397294 +0000 UTC m=+147.168602023" May 27 21:31:15.079680 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:15.079650 2564 scope.go:117] "RemoveContainer" containerID="62ba8701691a698f432ece42c92493f5e343140dadc33cb2f8bf9afcf7556e23" May 27 21:31:15.543239 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:15.543208 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-57bd96b888-nwvmp"] May 27 21:31:15.545449 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:15.545429 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-57bd96b888-nwvmp" May 27 21:31:15.547962 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:15.547936 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" May 27 21:31:15.548107 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:15.548091 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" May 27 21:31:15.548560 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:15.548544 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-j222k\"" May 27 21:31:15.560117 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:15.560095 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-57bd96b888-nwvmp"] May 27 21:31:15.583509 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:15.583479 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zj4v9\" (UniqueName: \"kubernetes.io/projected/87cf8e90-d191-4ca4-abb0-82f7f557b0e7-kube-api-access-zj4v9\") pod \"downloads-57bd96b888-nwvmp\" (UID: \"87cf8e90-d191-4ca4-abb0-82f7f557b0e7\") " pod="openshift-console/downloads-57bd96b888-nwvmp" May 27 21:31:15.679375 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:15.679350 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-d9gcd_16b8cf1e-1181-43fc-92e0-635b8523248d/console-operator/2.log" May 27 21:31:15.679506 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:15.679469 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-76dff48d59-d9gcd" event={"ID":"16b8cf1e-1181-43fc-92e0-635b8523248d","Type":"ContainerStarted","Data":"73a7d05dacbc1ee2a28951bc4c56e43dcb83be271dd782f24bce73202114916c"} May 27 21:31:15.679896 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:15.679878 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-76dff48d59-d9gcd" May 27 21:31:15.684401 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:15.684373 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zj4v9\" (UniqueName: \"kubernetes.io/projected/87cf8e90-d191-4ca4-abb0-82f7f557b0e7-kube-api-access-zj4v9\") pod \"downloads-57bd96b888-nwvmp\" (UID: \"87cf8e90-d191-4ca4-abb0-82f7f557b0e7\") " pod="openshift-console/downloads-57bd96b888-nwvmp" May 27 21:31:15.684807 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:15.684784 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-76dff48d59-d9gcd" May 27 21:31:15.694109 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:15.694089 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zj4v9\" (UniqueName: \"kubernetes.io/projected/87cf8e90-d191-4ca4-abb0-82f7f557b0e7-kube-api-access-zj4v9\") pod \"downloads-57bd96b888-nwvmp\" (UID: \"87cf8e90-d191-4ca4-abb0-82f7f557b0e7\") " pod="openshift-console/downloads-57bd96b888-nwvmp" May 27 21:31:15.702128 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:15.702088 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console-operator/console-operator-76dff48d59-d9gcd" podStartSLOduration=52.178520058 podStartE2EDuration="54.702077434s" podCreationTimestamp="2026-05-27 21:30:21 +0000 UTC" firstStartedPulling="2026-05-27 21:30:21.53622806 +0000 UTC m=+95.010432770" lastFinishedPulling="2026-05-27 21:30:24.059785435 +0000 UTC m=+97.533990146" observedRunningTime="2026-05-27 21:31:15.700737075 +0000 UTC m=+149.174941805" watchObservedRunningTime="2026-05-27 21:31:15.702077434 +0000 UTC m=+149.176282163" May 27 21:31:15.855202 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:15.855130 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-57bd96b888-nwvmp" May 27 21:31:15.992467 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:15.992438 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-57bd96b888-nwvmp"] May 27 21:31:15.995657 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:31:15.995627 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87cf8e90_d191_4ca4_abb0_82f7f557b0e7.slice/crio-7ecfcc42cae37322660c95f3d33e6dddc44b526d04f4781f0a1a6a91c7625149 WatchSource:0}: Error finding container 7ecfcc42cae37322660c95f3d33e6dddc44b526d04f4781f0a1a6a91c7625149: Status 404 returned error can't find the container with id 7ecfcc42cae37322660c95f3d33e6dddc44b526d04f4781f0a1a6a91c7625149 May 27 21:31:16.685061 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:16.685002 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-57bd96b888-nwvmp" event={"ID":"87cf8e90-d191-4ca4-abb0-82f7f557b0e7","Type":"ContainerStarted","Data":"7ecfcc42cae37322660c95f3d33e6dddc44b526d04f4781f0a1a6a91c7625149"} May 27 21:31:17.601384 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:17.601355 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7c87c4fbbc-psmsw" May 27 21:31:22.960405 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:31:22.960361 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-4t29f" podUID="f0c8b7dd-e25c-44d5-a1ac-52e4cef19011" May 27 21:31:22.974529 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:31:22.974488 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-5ntj8" podUID="0a23d452-20fc-4665-9d3a-ef17d210a8d9" May 27 21:31:23.706118 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:23.706086 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4t29f" May 27 21:31:25.663922 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:25.663881 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-688d5c9664-k6972"] May 27 21:31:25.666617 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:25.666593 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-688d5c9664-k6972" May 27 21:31:25.670711 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:25.670608 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" May 27 21:31:25.670711 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:25.670631 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" May 27 21:31:25.670711 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:25.670631 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" May 27 21:31:25.670711 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:25.670706 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" May 27 21:31:25.671016 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:25.670868 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" May 27 21:31:25.671016 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:25.670970 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-s454j\"" May 27 21:31:25.677387 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:25.677362 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-688d5c9664-k6972"] May 27 21:31:25.781558 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:25.781526 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/eddc7232-f96e-4302-b0e2-f9aa284e20c1-service-ca\") pod \"console-688d5c9664-k6972\" (UID: \"eddc7232-f96e-4302-b0e2-f9aa284e20c1\") " pod="openshift-console/console-688d5c9664-k6972" May 27 21:31:25.781725 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:25.781571 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/eddc7232-f96e-4302-b0e2-f9aa284e20c1-console-serving-cert\") pod \"console-688d5c9664-k6972\" (UID: \"eddc7232-f96e-4302-b0e2-f9aa284e20c1\") " pod="openshift-console/console-688d5c9664-k6972" May 27 21:31:25.781725 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:25.781616 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/eddc7232-f96e-4302-b0e2-f9aa284e20c1-console-oauth-config\") pod \"console-688d5c9664-k6972\" (UID: \"eddc7232-f96e-4302-b0e2-f9aa284e20c1\") " pod="openshift-console/console-688d5c9664-k6972" May 27 21:31:25.781725 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:25.781651 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9gn9\" (UniqueName: \"kubernetes.io/projected/eddc7232-f96e-4302-b0e2-f9aa284e20c1-kube-api-access-d9gn9\") pod \"console-688d5c9664-k6972\" (UID: \"eddc7232-f96e-4302-b0e2-f9aa284e20c1\") " pod="openshift-console/console-688d5c9664-k6972" May 27 21:31:25.781856 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:25.781757 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/eddc7232-f96e-4302-b0e2-f9aa284e20c1-console-config\") pod \"console-688d5c9664-k6972\" (UID: \"eddc7232-f96e-4302-b0e2-f9aa284e20c1\") " pod="openshift-console/console-688d5c9664-k6972" May 27 21:31:25.781856 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:25.781787 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/eddc7232-f96e-4302-b0e2-f9aa284e20c1-oauth-serving-cert\") pod \"console-688d5c9664-k6972\" (UID: \"eddc7232-f96e-4302-b0e2-f9aa284e20c1\") " pod="openshift-console/console-688d5c9664-k6972" May 27 21:31:25.882873 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:25.882833 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/eddc7232-f96e-4302-b0e2-f9aa284e20c1-console-config\") pod \"console-688d5c9664-k6972\" (UID: \"eddc7232-f96e-4302-b0e2-f9aa284e20c1\") " pod="openshift-console/console-688d5c9664-k6972" May 27 21:31:25.883042 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:25.882880 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/eddc7232-f96e-4302-b0e2-f9aa284e20c1-oauth-serving-cert\") pod \"console-688d5c9664-k6972\" (UID: \"eddc7232-f96e-4302-b0e2-f9aa284e20c1\") " pod="openshift-console/console-688d5c9664-k6972" May 27 21:31:25.883042 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:25.882929 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/eddc7232-f96e-4302-b0e2-f9aa284e20c1-service-ca\") pod \"console-688d5c9664-k6972\" (UID: \"eddc7232-f96e-4302-b0e2-f9aa284e20c1\") " pod="openshift-console/console-688d5c9664-k6972" May 27 21:31:25.883042 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:25.882950 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/eddc7232-f96e-4302-b0e2-f9aa284e20c1-console-serving-cert\") pod \"console-688d5c9664-k6972\" (UID: \"eddc7232-f96e-4302-b0e2-f9aa284e20c1\") " pod="openshift-console/console-688d5c9664-k6972" May 27 21:31:25.883042 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:25.882985 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/eddc7232-f96e-4302-b0e2-f9aa284e20c1-console-oauth-config\") pod \"console-688d5c9664-k6972\" (UID: \"eddc7232-f96e-4302-b0e2-f9aa284e20c1\") " pod="openshift-console/console-688d5c9664-k6972" May 27 21:31:25.883042 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:25.883027 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-d9gn9\" (UniqueName: \"kubernetes.io/projected/eddc7232-f96e-4302-b0e2-f9aa284e20c1-kube-api-access-d9gn9\") pod \"console-688d5c9664-k6972\" (UID: \"eddc7232-f96e-4302-b0e2-f9aa284e20c1\") " pod="openshift-console/console-688d5c9664-k6972" May 27 21:31:25.883625 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:25.883570 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/eddc7232-f96e-4302-b0e2-f9aa284e20c1-console-config\") pod \"console-688d5c9664-k6972\" (UID: \"eddc7232-f96e-4302-b0e2-f9aa284e20c1\") " pod="openshift-console/console-688d5c9664-k6972" May 27 21:31:25.883994 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:25.883972 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/eddc7232-f96e-4302-b0e2-f9aa284e20c1-service-ca\") pod \"console-688d5c9664-k6972\" (UID: \"eddc7232-f96e-4302-b0e2-f9aa284e20c1\") " pod="openshift-console/console-688d5c9664-k6972" May 27 21:31:25.884078 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:25.884046 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/eddc7232-f96e-4302-b0e2-f9aa284e20c1-oauth-serving-cert\") pod \"console-688d5c9664-k6972\" (UID: \"eddc7232-f96e-4302-b0e2-f9aa284e20c1\") " pod="openshift-console/console-688d5c9664-k6972" May 27 21:31:25.886037 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:25.886010 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/eddc7232-f96e-4302-b0e2-f9aa284e20c1-console-serving-cert\") pod \"console-688d5c9664-k6972\" (UID: \"eddc7232-f96e-4302-b0e2-f9aa284e20c1\") " pod="openshift-console/console-688d5c9664-k6972" May 27 21:31:25.886431 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:25.886411 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/eddc7232-f96e-4302-b0e2-f9aa284e20c1-console-oauth-config\") pod \"console-688d5c9664-k6972\" (UID: \"eddc7232-f96e-4302-b0e2-f9aa284e20c1\") " pod="openshift-console/console-688d5c9664-k6972" May 27 21:31:25.892129 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:25.892082 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9gn9\" (UniqueName: \"kubernetes.io/projected/eddc7232-f96e-4302-b0e2-f9aa284e20c1-kube-api-access-d9gn9\") pod \"console-688d5c9664-k6972\" (UID: \"eddc7232-f96e-4302-b0e2-f9aa284e20c1\") " pod="openshift-console/console-688d5c9664-k6972" May 27 21:31:25.979784 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:25.979738 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-688d5c9664-k6972" May 27 21:31:26.126459 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:26.126424 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-688d5c9664-k6972"] May 27 21:31:26.131579 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:31:26.131540 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeddc7232_f96e_4302_b0e2_f9aa284e20c1.slice/crio-ae96d5a930555e15f66fe7a29e743763e1e3b81b8e198c316c7cd27015fec83b WatchSource:0}: Error finding container ae96d5a930555e15f66fe7a29e743763e1e3b81b8e198c316c7cd27015fec83b: Status 404 returned error can't find the container with id ae96d5a930555e15f66fe7a29e743763e1e3b81b8e198c316c7cd27015fec83b May 27 21:31:26.716181 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:26.716145 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-688d5c9664-k6972" event={"ID":"eddc7232-f96e-4302-b0e2-f9aa284e20c1","Type":"ContainerStarted","Data":"ae96d5a930555e15f66fe7a29e743763e1e3b81b8e198c316c7cd27015fec83b"} May 27 21:31:27.799948 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:27.799909 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f0c8b7dd-e25c-44d5-a1ac-52e4cef19011-metrics-tls\") pod \"dns-default-4t29f\" (UID: \"f0c8b7dd-e25c-44d5-a1ac-52e4cef19011\") " pod="openshift-dns/dns-default-4t29f" May 27 21:31:27.802473 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:27.802442 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f0c8b7dd-e25c-44d5-a1ac-52e4cef19011-metrics-tls\") pod \"dns-default-4t29f\" (UID: \"f0c8b7dd-e25c-44d5-a1ac-52e4cef19011\") " pod="openshift-dns/dns-default-4t29f" May 27 21:31:27.900954 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:27.900912 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0a23d452-20fc-4665-9d3a-ef17d210a8d9-cert\") pod \"ingress-canary-5ntj8\" (UID: \"0a23d452-20fc-4665-9d3a-ef17d210a8d9\") " pod="openshift-ingress-canary/ingress-canary-5ntj8" May 27 21:31:27.903494 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:27.903470 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0a23d452-20fc-4665-9d3a-ef17d210a8d9-cert\") pod \"ingress-canary-5ntj8\" (UID: \"0a23d452-20fc-4665-9d3a-ef17d210a8d9\") " pod="openshift-ingress-canary/ingress-canary-5ntj8" May 27 21:31:27.910532 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:27.910511 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xztv5\"" May 27 21:31:27.917844 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:27.917828 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4t29f" May 27 21:31:33.030387 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:33.030357 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4t29f"] May 27 21:31:33.032959 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:31:33.032931 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0c8b7dd_e25c_44d5_a1ac_52e4cef19011.slice/crio-c68bf24e2e2d3ba8ac682414ef9a82a1fedd47573d1a6f179bcd61b862e62b96 WatchSource:0}: Error finding container c68bf24e2e2d3ba8ac682414ef9a82a1fedd47573d1a6f179bcd61b862e62b96: Status 404 returned error can't find the container with id c68bf24e2e2d3ba8ac682414ef9a82a1fedd47573d1a6f179bcd61b862e62b96 May 27 21:31:33.555709 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:33.555668 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-54fd6bc5c4-77nkf_030b9490-f0b8-4a41-90a9-da20af4103be/kube-state-metrics/0.log" May 27 21:31:33.738785 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:33.738715 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4t29f" event={"ID":"f0c8b7dd-e25c-44d5-a1ac-52e4cef19011","Type":"ContainerStarted","Data":"c68bf24e2e2d3ba8ac682414ef9a82a1fedd47573d1a6f179bcd61b862e62b96"} May 27 21:31:33.740954 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:33.740914 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-57bd96b888-nwvmp" event={"ID":"87cf8e90-d191-4ca4-abb0-82f7f557b0e7","Type":"ContainerStarted","Data":"37da72209c7f9759c9cd1bb19ee4ab766a74614a88dadf9e465a8636eed424dd"} May 27 21:31:33.741509 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:33.741473 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-57bd96b888-nwvmp" May 27 21:31:33.754116 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:33.754094 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-57bd96b888-nwvmp" May 27 21:31:33.755259 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:33.755239 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-54fd6bc5c4-77nkf_030b9490-f0b8-4a41-90a9-da20af4103be/kube-rbac-proxy-main/0.log" May 27 21:31:33.772194 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:33.772118 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/downloads-57bd96b888-nwvmp" podStartSLOduration=1.79992424 podStartE2EDuration="18.772101556s" podCreationTimestamp="2026-05-27 21:31:15 +0000 UTC" firstStartedPulling="2026-05-27 21:31:15.997475584 +0000 UTC m=+149.471680292" lastFinishedPulling="2026-05-27 21:31:32.969652894 +0000 UTC m=+166.443857608" observedRunningTime="2026-05-27 21:31:33.770549876 +0000 UTC m=+167.244754643" watchObservedRunningTime="2026-05-27 21:31:33.772101556 +0000 UTC m=+167.246306287" May 27 21:31:33.953997 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:33.953964 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-54fd6bc5c4-77nkf_030b9490-f0b8-4a41-90a9-da20af4103be/kube-rbac-proxy-self/0.log" May 27 21:31:34.260750 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:34.260669 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-6f77bb86f7-wcgw4"] May 27 21:31:34.274011 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:34.273392 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f77bb86f7-wcgw4" May 27 21:31:34.282823 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:34.282788 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f77bb86f7-wcgw4"] May 27 21:31:34.290806 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:34.290783 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" May 27 21:31:34.354842 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:34.354783 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-665c9c8584-xshg4_b7a8fe62-9758-4449-b720-78d9c0842165/monitoring-plugin/0.log" May 27 21:31:34.361366 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:34.361001 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/de626df4-dde7-4bf7-b78b-1a6fdd7ab763-console-config\") pod \"console-6f77bb86f7-wcgw4\" (UID: \"de626df4-dde7-4bf7-b78b-1a6fdd7ab763\") " pod="openshift-console/console-6f77bb86f7-wcgw4" May 27 21:31:34.361366 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:34.361046 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/de626df4-dde7-4bf7-b78b-1a6fdd7ab763-oauth-serving-cert\") pod \"console-6f77bb86f7-wcgw4\" (UID: \"de626df4-dde7-4bf7-b78b-1a6fdd7ab763\") " pod="openshift-console/console-6f77bb86f7-wcgw4" May 27 21:31:34.361366 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:34.361145 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/de626df4-dde7-4bf7-b78b-1a6fdd7ab763-console-oauth-config\") pod \"console-6f77bb86f7-wcgw4\" (UID: \"de626df4-dde7-4bf7-b78b-1a6fdd7ab763\") " pod="openshift-console/console-6f77bb86f7-wcgw4" May 27 21:31:34.361366 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:34.361175 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s852n\" (UniqueName: \"kubernetes.io/projected/de626df4-dde7-4bf7-b78b-1a6fdd7ab763-kube-api-access-s852n\") pod \"console-6f77bb86f7-wcgw4\" (UID: \"de626df4-dde7-4bf7-b78b-1a6fdd7ab763\") " pod="openshift-console/console-6f77bb86f7-wcgw4" May 27 21:31:34.361366 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:34.361202 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/de626df4-dde7-4bf7-b78b-1a6fdd7ab763-service-ca\") pod \"console-6f77bb86f7-wcgw4\" (UID: \"de626df4-dde7-4bf7-b78b-1a6fdd7ab763\") " pod="openshift-console/console-6f77bb86f7-wcgw4" May 27 21:31:34.361366 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:34.361232 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de626df4-dde7-4bf7-b78b-1a6fdd7ab763-trusted-ca-bundle\") pod \"console-6f77bb86f7-wcgw4\" (UID: \"de626df4-dde7-4bf7-b78b-1a6fdd7ab763\") " pod="openshift-console/console-6f77bb86f7-wcgw4" May 27 21:31:34.361366 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:34.361291 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/de626df4-dde7-4bf7-b78b-1a6fdd7ab763-console-serving-cert\") pod \"console-6f77bb86f7-wcgw4\" (UID: \"de626df4-dde7-4bf7-b78b-1a6fdd7ab763\") " pod="openshift-console/console-6f77bb86f7-wcgw4" May 27 21:31:34.462722 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:34.462683 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/de626df4-dde7-4bf7-b78b-1a6fdd7ab763-console-serving-cert\") pod \"console-6f77bb86f7-wcgw4\" (UID: \"de626df4-dde7-4bf7-b78b-1a6fdd7ab763\") " pod="openshift-console/console-6f77bb86f7-wcgw4" May 27 21:31:34.462905 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:34.462761 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/de626df4-dde7-4bf7-b78b-1a6fdd7ab763-console-config\") pod \"console-6f77bb86f7-wcgw4\" (UID: \"de626df4-dde7-4bf7-b78b-1a6fdd7ab763\") " pod="openshift-console/console-6f77bb86f7-wcgw4" May 27 21:31:34.462905 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:34.462790 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/de626df4-dde7-4bf7-b78b-1a6fdd7ab763-oauth-serving-cert\") pod \"console-6f77bb86f7-wcgw4\" (UID: \"de626df4-dde7-4bf7-b78b-1a6fdd7ab763\") " pod="openshift-console/console-6f77bb86f7-wcgw4" May 27 21:31:34.462905 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:34.462864 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/de626df4-dde7-4bf7-b78b-1a6fdd7ab763-console-oauth-config\") pod \"console-6f77bb86f7-wcgw4\" (UID: \"de626df4-dde7-4bf7-b78b-1a6fdd7ab763\") " pod="openshift-console/console-6f77bb86f7-wcgw4" May 27 21:31:34.462905 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:34.462898 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-s852n\" (UniqueName: \"kubernetes.io/projected/de626df4-dde7-4bf7-b78b-1a6fdd7ab763-kube-api-access-s852n\") pod \"console-6f77bb86f7-wcgw4\" (UID: \"de626df4-dde7-4bf7-b78b-1a6fdd7ab763\") " pod="openshift-console/console-6f77bb86f7-wcgw4" May 27 21:31:34.463134 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:34.462929 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/de626df4-dde7-4bf7-b78b-1a6fdd7ab763-service-ca\") pod \"console-6f77bb86f7-wcgw4\" (UID: \"de626df4-dde7-4bf7-b78b-1a6fdd7ab763\") " pod="openshift-console/console-6f77bb86f7-wcgw4" May 27 21:31:34.463134 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:34.462960 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de626df4-dde7-4bf7-b78b-1a6fdd7ab763-trusted-ca-bundle\") pod \"console-6f77bb86f7-wcgw4\" (UID: \"de626df4-dde7-4bf7-b78b-1a6fdd7ab763\") " pod="openshift-console/console-6f77bb86f7-wcgw4" May 27 21:31:34.465403 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:34.463711 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/de626df4-dde7-4bf7-b78b-1a6fdd7ab763-oauth-serving-cert\") pod \"console-6f77bb86f7-wcgw4\" (UID: \"de626df4-dde7-4bf7-b78b-1a6fdd7ab763\") " pod="openshift-console/console-6f77bb86f7-wcgw4" May 27 21:31:34.465403 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:34.463851 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de626df4-dde7-4bf7-b78b-1a6fdd7ab763-trusted-ca-bundle\") pod \"console-6f77bb86f7-wcgw4\" (UID: \"de626df4-dde7-4bf7-b78b-1a6fdd7ab763\") " pod="openshift-console/console-6f77bb86f7-wcgw4" May 27 21:31:34.465403 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:34.464666 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/de626df4-dde7-4bf7-b78b-1a6fdd7ab763-console-config\") pod \"console-6f77bb86f7-wcgw4\" (UID: \"de626df4-dde7-4bf7-b78b-1a6fdd7ab763\") " pod="openshift-console/console-6f77bb86f7-wcgw4" May 27 21:31:34.465666 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:34.465495 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/de626df4-dde7-4bf7-b78b-1a6fdd7ab763-service-ca\") pod \"console-6f77bb86f7-wcgw4\" (UID: \"de626df4-dde7-4bf7-b78b-1a6fdd7ab763\") " pod="openshift-console/console-6f77bb86f7-wcgw4" May 27 21:31:34.468286 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:34.468187 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/de626df4-dde7-4bf7-b78b-1a6fdd7ab763-console-serving-cert\") pod \"console-6f77bb86f7-wcgw4\" (UID: \"de626df4-dde7-4bf7-b78b-1a6fdd7ab763\") " pod="openshift-console/console-6f77bb86f7-wcgw4" May 27 21:31:34.468657 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:34.468617 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/de626df4-dde7-4bf7-b78b-1a6fdd7ab763-console-oauth-config\") pod \"console-6f77bb86f7-wcgw4\" (UID: \"de626df4-dde7-4bf7-b78b-1a6fdd7ab763\") " pod="openshift-console/console-6f77bb86f7-wcgw4" May 27 21:31:34.481995 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:34.481944 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s852n\" (UniqueName: \"kubernetes.io/projected/de626df4-dde7-4bf7-b78b-1a6fdd7ab763-kube-api-access-s852n\") pod \"console-6f77bb86f7-wcgw4\" (UID: \"de626df4-dde7-4bf7-b78b-1a6fdd7ab763\") " pod="openshift-console/console-6f77bb86f7-wcgw4" May 27 21:31:34.588840 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:34.588768 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f77bb86f7-wcgw4" May 27 21:31:35.084655 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:35.084624 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5ntj8" May 27 21:31:35.087603 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:35.087577 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-vmd24\"" May 27 21:31:35.095889 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:35.095837 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5ntj8" May 27 21:31:35.154753 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:35.154706 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8prph_32f3075a-ac99-468a-a2b9-c76f16897682/init-textfile/0.log" May 27 21:31:35.354779 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:35.354554 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8prph_32f3075a-ac99-468a-a2b9-c76f16897682/node-exporter/0.log" May 27 21:31:35.556761 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:35.556701 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8prph_32f3075a-ac99-468a-a2b9-c76f16897682/kube-rbac-proxy/0.log" May 27 21:31:37.047532 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:37.047172 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5ntj8"] May 27 21:31:37.050635 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:31:37.050585 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a23d452_20fc_4665_9d3a_ef17d210a8d9.slice/crio-01d050165ebb608a03364493f6a623c3fa1580f424e35def8c63f89afa63d1e0 WatchSource:0}: Error finding container 01d050165ebb608a03364493f6a623c3fa1580f424e35def8c63f89afa63d1e0: Status 404 returned error can't find the container with id 01d050165ebb608a03364493f6a623c3fa1580f424e35def8c63f89afa63d1e0 May 27 21:31:37.072278 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:37.072232 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f77bb86f7-wcgw4"] May 27 21:31:37.079305 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:31:37.079260 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde626df4_dde7_4bf7_b78b_1a6fdd7ab763.slice/crio-544c4a777237919c2b5418e1800935b601bb0689fc8c8b56186478e44cd40a16 WatchSource:0}: Error finding container 544c4a777237919c2b5418e1800935b601bb0689fc8c8b56186478e44cd40a16: Status 404 returned error can't find the container with id 544c4a777237919c2b5418e1800935b601bb0689fc8c8b56186478e44cd40a16 May 27 21:31:37.755681 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:37.755636 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5ntj8" event={"ID":"0a23d452-20fc-4665-9d3a-ef17d210a8d9","Type":"ContainerStarted","Data":"01d050165ebb608a03364493f6a623c3fa1580f424e35def8c63f89afa63d1e0"} May 27 21:31:37.757646 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:37.757602 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f77bb86f7-wcgw4" event={"ID":"de626df4-dde7-4bf7-b78b-1a6fdd7ab763","Type":"ContainerStarted","Data":"043161d01ef6c663f9605bf562a48ee144cc18e85a9458aa8408805504e6bcae"} May 27 21:31:37.757779 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:37.757668 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f77bb86f7-wcgw4" event={"ID":"de626df4-dde7-4bf7-b78b-1a6fdd7ab763","Type":"ContainerStarted","Data":"544c4a777237919c2b5418e1800935b601bb0689fc8c8b56186478e44cd40a16"} May 27 21:31:37.760252 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:37.760224 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4t29f" event={"ID":"f0c8b7dd-e25c-44d5-a1ac-52e4cef19011","Type":"ContainerStarted","Data":"5e9f117a852b02441467c0bf6f562d092e27111910c6b227402c7ffd7e571329"} May 27 21:31:37.760367 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:37.760286 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4t29f" event={"ID":"f0c8b7dd-e25c-44d5-a1ac-52e4cef19011","Type":"ContainerStarted","Data":"096a127c78f047d2b7e18368141eaaccc63ea3a75f0a35e489d52635bca73193"} May 27 21:31:37.760635 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:37.760472 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-4t29f" May 27 21:31:37.762246 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:37.762220 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-688d5c9664-k6972" event={"ID":"eddc7232-f96e-4302-b0e2-f9aa284e20c1","Type":"ContainerStarted","Data":"263aba5ee06a8e7a661df666e21c611cb9cf7ebb749ee88fbe59bf5b95a780ed"} May 27 21:31:37.800834 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:37.800730 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-4t29f" podStartSLOduration=135.029349969 podStartE2EDuration="2m18.800710999s" podCreationTimestamp="2026-05-27 21:29:19 +0000 UTC" firstStartedPulling="2026-05-27 21:31:33.034805332 +0000 UTC m=+166.509010041" lastFinishedPulling="2026-05-27 21:31:36.806166357 +0000 UTC m=+170.280371071" observedRunningTime="2026-05-27 21:31:37.799569215 +0000 UTC m=+171.273773946" watchObservedRunningTime="2026-05-27 21:31:37.800710999 +0000 UTC m=+171.274915731" May 27 21:31:37.801936 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:37.801820 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-6f77bb86f7-wcgw4" podStartSLOduration=3.80170487 podStartE2EDuration="3.80170487s" podCreationTimestamp="2026-05-27 21:31:34 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-27 21:31:37.779606493 +0000 UTC m=+171.253811235" watchObservedRunningTime="2026-05-27 21:31:37.80170487 +0000 UTC m=+171.275909600" May 27 21:31:37.822622 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:37.822549 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-688d5c9664-k6972" podStartSLOduration=2.142449835 podStartE2EDuration="12.822524408s" podCreationTimestamp="2026-05-27 21:31:25 +0000 UTC" firstStartedPulling="2026-05-27 21:31:26.133628896 +0000 UTC m=+159.607833604" lastFinishedPulling="2026-05-27 21:31:36.813703455 +0000 UTC m=+170.287908177" observedRunningTime="2026-05-27 21:31:37.820813626 +0000 UTC m=+171.295018391" watchObservedRunningTime="2026-05-27 21:31:37.822524408 +0000 UTC m=+171.296729140" May 27 21:31:38.355482 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:38.355447 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7779bbdd78-jhdrv_ece3e9ae-4d45-45d6-a435-df9cb81ebb42/prometheus-operator/0.log" May 27 21:31:38.554232 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:38.554202 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7779bbdd78-jhdrv_ece3e9ae-4d45-45d6-a435-df9cb81ebb42/kube-rbac-proxy/0.log" May 27 21:31:38.754010 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:38.753977 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-6c4bbc67b7-jq28q_4da4d84a-f34f-44d3-9312-a8888ac22fa3/prometheus-operator-admission-webhook/0.log" May 27 21:31:39.770890 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:39.770849 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5ntj8" event={"ID":"0a23d452-20fc-4665-9d3a-ef17d210a8d9","Type":"ContainerStarted","Data":"ad3f2021dbbf18f324794596464598921ba00b1f580af237298464b1c62982bb"} May 27 21:31:39.791858 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:39.791802 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-5ntj8" podStartSLOduration=138.244201121 podStartE2EDuration="2m20.791786327s" podCreationTimestamp="2026-05-27 21:29:19 +0000 UTC" firstStartedPulling="2026-05-27 21:31:37.053970994 +0000 UTC m=+170.528175719" lastFinishedPulling="2026-05-27 21:31:39.601556218 +0000 UTC m=+173.075760925" observedRunningTime="2026-05-27 21:31:39.789916155 +0000 UTC m=+173.264120892" watchObservedRunningTime="2026-05-27 21:31:39.791786327 +0000 UTC m=+173.265991056" May 27 21:31:40.354812 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:40.354780 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-d9gcd_16b8cf1e-1181-43fc-92e0-635b8523248d/console-operator/2.log" May 27 21:31:40.556841 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:40.556807 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-d9gcd_16b8cf1e-1181-43fc-92e0-635b8523248d/console-operator/3.log" May 27 21:31:40.753773 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:40.753743 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-688d5c9664-k6972_eddc7232-f96e-4302-b0e2-f9aa284e20c1/console/0.log" May 27 21:31:40.955071 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:40.955036 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-57bd96b888-nwvmp_87cf8e90-d191-4ca4-abb0-82f7f557b0e7/download-server/0.log" May 27 21:31:41.154816 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:41.154736 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-688f8b8554-fn4zx_51660acb-022f-4265-bec8-ae93f5315b17/router/0.log" May 27 21:31:41.353521 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:41.353486 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5ntj8_0a23d452-20fc-4665-9d3a-ef17d210a8d9/serve-healthcheck-canary/0.log" May 27 21:31:44.589773 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:44.589736 2564 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6f77bb86f7-wcgw4" May 27 21:31:44.589773 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:44.589776 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6f77bb86f7-wcgw4" May 27 21:31:44.594643 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:44.594622 2564 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6f77bb86f7-wcgw4" May 27 21:31:44.791742 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:44.791712 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6f77bb86f7-wcgw4" May 27 21:31:44.874348 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:44.873755 2564 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-688d5c9664-k6972"] May 27 21:31:45.980032 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:45.979998 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-688d5c9664-k6972" May 27 21:31:47.768620 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:47.768588 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-4t29f" May 27 21:31:49.803639 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:49.803547 2564 generic.go:358] "Generic (PLEG): container finished" podID="1df7cdcd-3a57-4d1a-97d4-2638a9f49055" containerID="615a4c170ae95094dee992d7973e02dabbadd3b1625eeddfb2bb31802e0716bc" exitCode=0 May 27 21:31:49.803639 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:49.803597 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-z76cl" event={"ID":"1df7cdcd-3a57-4d1a-97d4-2638a9f49055","Type":"ContainerDied","Data":"615a4c170ae95094dee992d7973e02dabbadd3b1625eeddfb2bb31802e0716bc"} May 27 21:31:49.804026 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:49.803941 2564 scope.go:117] "RemoveContainer" containerID="615a4c170ae95094dee992d7973e02dabbadd3b1625eeddfb2bb31802e0716bc" May 27 21:31:50.807832 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:31:50.807797 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-z76cl" event={"ID":"1df7cdcd-3a57-4d1a-97d4-2638a9f49055","Type":"ContainerStarted","Data":"6f2919cb7d05dd4d68f94cce2d5fc3befe0daa24a33ca49cdba65967a3ac75da"} May 27 21:32:09.902547 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:09.902482 2564 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-688d5c9664-k6972" podUID="eddc7232-f96e-4302-b0e2-f9aa284e20c1" containerName="console" containerID="cri-o://263aba5ee06a8e7a661df666e21c611cb9cf7ebb749ee88fbe59bf5b95a780ed" gracePeriod=15 May 27 21:32:10.170680 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:10.170652 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-688d5c9664-k6972_eddc7232-f96e-4302-b0e2-f9aa284e20c1/console/0.log" May 27 21:32:10.170811 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:10.170738 2564 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-688d5c9664-k6972" May 27 21:32:10.270040 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:10.270012 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/eddc7232-f96e-4302-b0e2-f9aa284e20c1-oauth-serving-cert\") pod \"eddc7232-f96e-4302-b0e2-f9aa284e20c1\" (UID: \"eddc7232-f96e-4302-b0e2-f9aa284e20c1\") " May 27 21:32:10.270195 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:10.270080 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/eddc7232-f96e-4302-b0e2-f9aa284e20c1-console-serving-cert\") pod \"eddc7232-f96e-4302-b0e2-f9aa284e20c1\" (UID: \"eddc7232-f96e-4302-b0e2-f9aa284e20c1\") " May 27 21:32:10.270195 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:10.270102 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/eddc7232-f96e-4302-b0e2-f9aa284e20c1-console-oauth-config\") pod \"eddc7232-f96e-4302-b0e2-f9aa284e20c1\" (UID: \"eddc7232-f96e-4302-b0e2-f9aa284e20c1\") " May 27 21:32:10.270195 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:10.270123 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/eddc7232-f96e-4302-b0e2-f9aa284e20c1-service-ca\") pod \"eddc7232-f96e-4302-b0e2-f9aa284e20c1\" (UID: \"eddc7232-f96e-4302-b0e2-f9aa284e20c1\") " May 27 21:32:10.270195 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:10.270143 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d9gn9\" (UniqueName: \"kubernetes.io/projected/eddc7232-f96e-4302-b0e2-f9aa284e20c1-kube-api-access-d9gn9\") pod \"eddc7232-f96e-4302-b0e2-f9aa284e20c1\" (UID: \"eddc7232-f96e-4302-b0e2-f9aa284e20c1\") " May 27 21:32:10.270195 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:10.270161 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/eddc7232-f96e-4302-b0e2-f9aa284e20c1-console-config\") pod \"eddc7232-f96e-4302-b0e2-f9aa284e20c1\" (UID: \"eddc7232-f96e-4302-b0e2-f9aa284e20c1\") " May 27 21:32:10.270586 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:10.270474 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eddc7232-f96e-4302-b0e2-f9aa284e20c1-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "eddc7232-f96e-4302-b0e2-f9aa284e20c1" (UID: "eddc7232-f96e-4302-b0e2-f9aa284e20c1"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 27 21:32:10.270691 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:10.270595 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eddc7232-f96e-4302-b0e2-f9aa284e20c1-service-ca" (OuterVolumeSpecName: "service-ca") pod "eddc7232-f96e-4302-b0e2-f9aa284e20c1" (UID: "eddc7232-f96e-4302-b0e2-f9aa284e20c1"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 27 21:32:10.270691 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:10.270656 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eddc7232-f96e-4302-b0e2-f9aa284e20c1-console-config" (OuterVolumeSpecName: "console-config") pod "eddc7232-f96e-4302-b0e2-f9aa284e20c1" (UID: "eddc7232-f96e-4302-b0e2-f9aa284e20c1"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 27 21:32:10.272495 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:10.272468 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eddc7232-f96e-4302-b0e2-f9aa284e20c1-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "eddc7232-f96e-4302-b0e2-f9aa284e20c1" (UID: "eddc7232-f96e-4302-b0e2-f9aa284e20c1"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 27 21:32:10.272495 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:10.272483 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eddc7232-f96e-4302-b0e2-f9aa284e20c1-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "eddc7232-f96e-4302-b0e2-f9aa284e20c1" (UID: "eddc7232-f96e-4302-b0e2-f9aa284e20c1"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 27 21:32:10.272495 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:10.272489 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eddc7232-f96e-4302-b0e2-f9aa284e20c1-kube-api-access-d9gn9" (OuterVolumeSpecName: "kube-api-access-d9gn9") pod "eddc7232-f96e-4302-b0e2-f9aa284e20c1" (UID: "eddc7232-f96e-4302-b0e2-f9aa284e20c1"). InnerVolumeSpecName "kube-api-access-d9gn9". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 27 21:32:10.371584 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:10.371557 2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/eddc7232-f96e-4302-b0e2-f9aa284e20c1-service-ca\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:32:10.371584 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:10.371582 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d9gn9\" (UniqueName: \"kubernetes.io/projected/eddc7232-f96e-4302-b0e2-f9aa284e20c1-kube-api-access-d9gn9\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:32:10.371748 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:10.371594 2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/eddc7232-f96e-4302-b0e2-f9aa284e20c1-console-config\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:32:10.371748 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:10.371604 2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/eddc7232-f96e-4302-b0e2-f9aa284e20c1-oauth-serving-cert\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:32:10.371748 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:10.371613 2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/eddc7232-f96e-4302-b0e2-f9aa284e20c1-console-serving-cert\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:32:10.371748 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:10.371621 2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/eddc7232-f96e-4302-b0e2-f9aa284e20c1-console-oauth-config\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:32:10.870399 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:10.870371 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-688d5c9664-k6972_eddc7232-f96e-4302-b0e2-f9aa284e20c1/console/0.log" May 27 21:32:10.870553 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:10.870416 2564 generic.go:358] "Generic (PLEG): container finished" podID="eddc7232-f96e-4302-b0e2-f9aa284e20c1" containerID="263aba5ee06a8e7a661df666e21c611cb9cf7ebb749ee88fbe59bf5b95a780ed" exitCode=2 May 27 21:32:10.870553 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:10.870492 2564 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-688d5c9664-k6972" May 27 21:32:10.870553 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:10.870505 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-688d5c9664-k6972" event={"ID":"eddc7232-f96e-4302-b0e2-f9aa284e20c1","Type":"ContainerDied","Data":"263aba5ee06a8e7a661df666e21c611cb9cf7ebb749ee88fbe59bf5b95a780ed"} May 27 21:32:10.870553 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:10.870545 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-688d5c9664-k6972" event={"ID":"eddc7232-f96e-4302-b0e2-f9aa284e20c1","Type":"ContainerDied","Data":"ae96d5a930555e15f66fe7a29e743763e1e3b81b8e198c316c7cd27015fec83b"} May 27 21:32:10.870682 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:10.870562 2564 scope.go:117] "RemoveContainer" containerID="263aba5ee06a8e7a661df666e21c611cb9cf7ebb749ee88fbe59bf5b95a780ed" May 27 21:32:10.879391 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:10.879373 2564 scope.go:117] "RemoveContainer" containerID="263aba5ee06a8e7a661df666e21c611cb9cf7ebb749ee88fbe59bf5b95a780ed" May 27 21:32:10.879636 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:32:10.879617 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"263aba5ee06a8e7a661df666e21c611cb9cf7ebb749ee88fbe59bf5b95a780ed\": container with ID starting with 263aba5ee06a8e7a661df666e21c611cb9cf7ebb749ee88fbe59bf5b95a780ed not found: ID does not exist" containerID="263aba5ee06a8e7a661df666e21c611cb9cf7ebb749ee88fbe59bf5b95a780ed" May 27 21:32:10.879686 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:10.879644 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"263aba5ee06a8e7a661df666e21c611cb9cf7ebb749ee88fbe59bf5b95a780ed"} err="failed to get container status \"263aba5ee06a8e7a661df666e21c611cb9cf7ebb749ee88fbe59bf5b95a780ed\": rpc error: code = NotFound desc = could not find container \"263aba5ee06a8e7a661df666e21c611cb9cf7ebb749ee88fbe59bf5b95a780ed\": container with ID starting with 263aba5ee06a8e7a661df666e21c611cb9cf7ebb749ee88fbe59bf5b95a780ed not found: ID does not exist" May 27 21:32:10.893538 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:10.893516 2564 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-688d5c9664-k6972"] May 27 21:32:10.897808 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:10.897784 2564 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-688d5c9664-k6972"] May 27 21:32:11.084075 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:11.084039 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eddc7232-f96e-4302-b0e2-f9aa284e20c1" path="/var/lib/kubelet/pods/eddc7232-f96e-4302-b0e2-f9aa284e20c1/volumes" May 27 21:32:34.505759 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:34.505719 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-69c9b4fcb4-jmngr"] May 27 21:32:34.506288 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:34.506010 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eddc7232-f96e-4302-b0e2-f9aa284e20c1" containerName="console" May 27 21:32:34.506288 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:34.506021 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="eddc7232-f96e-4302-b0e2-f9aa284e20c1" containerName="console" May 27 21:32:34.506288 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:34.506093 2564 memory_manager.go:356] "RemoveStaleState removing state" podUID="eddc7232-f96e-4302-b0e2-f9aa284e20c1" containerName="console" May 27 21:32:34.508102 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:34.508083 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69c9b4fcb4-jmngr" May 27 21:32:34.521444 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:34.521410 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-69c9b4fcb4-jmngr"] May 27 21:32:34.677474 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:34.677435 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/963c876d-fac5-46e6-90b4-63ba0cb87714-oauth-serving-cert\") pod \"console-69c9b4fcb4-jmngr\" (UID: \"963c876d-fac5-46e6-90b4-63ba0cb87714\") " pod="openshift-console/console-69c9b4fcb4-jmngr" May 27 21:32:34.677792 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:34.677757 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/963c876d-fac5-46e6-90b4-63ba0cb87714-trusted-ca-bundle\") pod \"console-69c9b4fcb4-jmngr\" (UID: \"963c876d-fac5-46e6-90b4-63ba0cb87714\") " pod="openshift-console/console-69c9b4fcb4-jmngr" May 27 21:32:34.677927 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:34.677914 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/963c876d-fac5-46e6-90b4-63ba0cb87714-console-oauth-config\") pod \"console-69c9b4fcb4-jmngr\" (UID: \"963c876d-fac5-46e6-90b4-63ba0cb87714\") " pod="openshift-console/console-69c9b4fcb4-jmngr" May 27 21:32:34.678041 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:34.678026 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/963c876d-fac5-46e6-90b4-63ba0cb87714-service-ca\") pod \"console-69c9b4fcb4-jmngr\" (UID: \"963c876d-fac5-46e6-90b4-63ba0cb87714\") " pod="openshift-console/console-69c9b4fcb4-jmngr" May 27 21:32:34.678298 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:34.678259 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/963c876d-fac5-46e6-90b4-63ba0cb87714-console-config\") pod \"console-69c9b4fcb4-jmngr\" (UID: \"963c876d-fac5-46e6-90b4-63ba0cb87714\") " pod="openshift-console/console-69c9b4fcb4-jmngr" May 27 21:32:34.678401 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:34.678380 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/963c876d-fac5-46e6-90b4-63ba0cb87714-console-serving-cert\") pod \"console-69c9b4fcb4-jmngr\" (UID: \"963c876d-fac5-46e6-90b4-63ba0cb87714\") " pod="openshift-console/console-69c9b4fcb4-jmngr" May 27 21:32:34.678464 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:34.678422 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffrf7\" (UniqueName: \"kubernetes.io/projected/963c876d-fac5-46e6-90b4-63ba0cb87714-kube-api-access-ffrf7\") pod \"console-69c9b4fcb4-jmngr\" (UID: \"963c876d-fac5-46e6-90b4-63ba0cb87714\") " pod="openshift-console/console-69c9b4fcb4-jmngr" May 27 21:32:34.779096 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:34.778998 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/963c876d-fac5-46e6-90b4-63ba0cb87714-console-config\") pod \"console-69c9b4fcb4-jmngr\" (UID: \"963c876d-fac5-46e6-90b4-63ba0cb87714\") " pod="openshift-console/console-69c9b4fcb4-jmngr" May 27 21:32:34.779096 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:34.779048 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/963c876d-fac5-46e6-90b4-63ba0cb87714-console-serving-cert\") pod \"console-69c9b4fcb4-jmngr\" (UID: \"963c876d-fac5-46e6-90b4-63ba0cb87714\") " pod="openshift-console/console-69c9b4fcb4-jmngr" May 27 21:32:34.779373 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:34.779185 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ffrf7\" (UniqueName: \"kubernetes.io/projected/963c876d-fac5-46e6-90b4-63ba0cb87714-kube-api-access-ffrf7\") pod \"console-69c9b4fcb4-jmngr\" (UID: \"963c876d-fac5-46e6-90b4-63ba0cb87714\") " pod="openshift-console/console-69c9b4fcb4-jmngr" May 27 21:32:34.779373 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:34.779253 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/963c876d-fac5-46e6-90b4-63ba0cb87714-oauth-serving-cert\") pod \"console-69c9b4fcb4-jmngr\" (UID: \"963c876d-fac5-46e6-90b4-63ba0cb87714\") " pod="openshift-console/console-69c9b4fcb4-jmngr" May 27 21:32:34.779373 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:34.779321 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/963c876d-fac5-46e6-90b4-63ba0cb87714-trusted-ca-bundle\") pod \"console-69c9b4fcb4-jmngr\" (UID: \"963c876d-fac5-46e6-90b4-63ba0cb87714\") " pod="openshift-console/console-69c9b4fcb4-jmngr" May 27 21:32:34.779373 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:34.779360 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/963c876d-fac5-46e6-90b4-63ba0cb87714-console-oauth-config\") pod \"console-69c9b4fcb4-jmngr\" (UID: \"963c876d-fac5-46e6-90b4-63ba0cb87714\") " pod="openshift-console/console-69c9b4fcb4-jmngr" May 27 21:32:34.779579 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:34.779389 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/963c876d-fac5-46e6-90b4-63ba0cb87714-service-ca\") pod \"console-69c9b4fcb4-jmngr\" (UID: \"963c876d-fac5-46e6-90b4-63ba0cb87714\") " pod="openshift-console/console-69c9b4fcb4-jmngr" May 27 21:32:34.779829 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:34.779791 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/963c876d-fac5-46e6-90b4-63ba0cb87714-console-config\") pod \"console-69c9b4fcb4-jmngr\" (UID: \"963c876d-fac5-46e6-90b4-63ba0cb87714\") " pod="openshift-console/console-69c9b4fcb4-jmngr" May 27 21:32:34.779951 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:34.779932 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/963c876d-fac5-46e6-90b4-63ba0cb87714-oauth-serving-cert\") pod \"console-69c9b4fcb4-jmngr\" (UID: \"963c876d-fac5-46e6-90b4-63ba0cb87714\") " pod="openshift-console/console-69c9b4fcb4-jmngr" May 27 21:32:34.780096 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:34.780076 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/963c876d-fac5-46e6-90b4-63ba0cb87714-service-ca\") pod \"console-69c9b4fcb4-jmngr\" (UID: \"963c876d-fac5-46e6-90b4-63ba0cb87714\") " pod="openshift-console/console-69c9b4fcb4-jmngr" May 27 21:32:34.780311 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:34.780284 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/963c876d-fac5-46e6-90b4-63ba0cb87714-trusted-ca-bundle\") pod \"console-69c9b4fcb4-jmngr\" (UID: \"963c876d-fac5-46e6-90b4-63ba0cb87714\") " pod="openshift-console/console-69c9b4fcb4-jmngr" May 27 21:32:34.781713 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:34.781693 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/963c876d-fac5-46e6-90b4-63ba0cb87714-console-serving-cert\") pod \"console-69c9b4fcb4-jmngr\" (UID: \"963c876d-fac5-46e6-90b4-63ba0cb87714\") " pod="openshift-console/console-69c9b4fcb4-jmngr" May 27 21:32:34.781817 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:34.781798 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/963c876d-fac5-46e6-90b4-63ba0cb87714-console-oauth-config\") pod \"console-69c9b4fcb4-jmngr\" (UID: \"963c876d-fac5-46e6-90b4-63ba0cb87714\") " pod="openshift-console/console-69c9b4fcb4-jmngr" May 27 21:32:34.789135 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:34.789110 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffrf7\" (UniqueName: \"kubernetes.io/projected/963c876d-fac5-46e6-90b4-63ba0cb87714-kube-api-access-ffrf7\") pod \"console-69c9b4fcb4-jmngr\" (UID: \"963c876d-fac5-46e6-90b4-63ba0cb87714\") " pod="openshift-console/console-69c9b4fcb4-jmngr" May 27 21:32:34.818118 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:34.818078 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69c9b4fcb4-jmngr" May 27 21:32:34.949526 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:34.949496 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-69c9b4fcb4-jmngr"] May 27 21:32:34.952564 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:32:34.952532 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod963c876d_fac5_46e6_90b4_63ba0cb87714.slice/crio-777449a0bed9903ffc2327a763984e4a5de4fee071559fb267ab85c18e6649d7 WatchSource:0}: Error finding container 777449a0bed9903ffc2327a763984e4a5de4fee071559fb267ab85c18e6649d7: Status 404 returned error can't find the container with id 777449a0bed9903ffc2327a763984e4a5de4fee071559fb267ab85c18e6649d7 May 27 21:32:35.941519 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:35.941482 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69c9b4fcb4-jmngr" event={"ID":"963c876d-fac5-46e6-90b4-63ba0cb87714","Type":"ContainerStarted","Data":"fa659a29bdeb10644f68d7a1375aeaac1bf58f5f04b708d79738f33e1beeda67"} May 27 21:32:35.941519 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:35.941520 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69c9b4fcb4-jmngr" event={"ID":"963c876d-fac5-46e6-90b4-63ba0cb87714","Type":"ContainerStarted","Data":"777449a0bed9903ffc2327a763984e4a5de4fee071559fb267ab85c18e6649d7"} May 27 21:32:35.960120 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:35.960074 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-69c9b4fcb4-jmngr" podStartSLOduration=1.960060162 podStartE2EDuration="1.960060162s" podCreationTimestamp="2026-05-27 21:32:34 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-27 21:32:35.958870677 +0000 UTC m=+229.433075419" watchObservedRunningTime="2026-05-27 21:32:35.960060162 +0000 UTC m=+229.434264892" May 27 21:32:44.818753 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:44.818705 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-69c9b4fcb4-jmngr" May 27 21:32:44.819397 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:44.818765 2564 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-69c9b4fcb4-jmngr" May 27 21:32:44.823523 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:44.823499 2564 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-69c9b4fcb4-jmngr" May 27 21:32:44.971510 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:44.971482 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-69c9b4fcb4-jmngr" May 27 21:32:45.028851 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:32:45.028816 2564 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f77bb86f7-wcgw4"] May 27 21:33:10.052899 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:33:10.052776 2564 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-6f77bb86f7-wcgw4" podUID="de626df4-dde7-4bf7-b78b-1a6fdd7ab763" containerName="console" containerID="cri-o://043161d01ef6c663f9605bf562a48ee144cc18e85a9458aa8408805504e6bcae" gracePeriod=15 May 27 21:33:10.290730 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:33:10.290699 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6f77bb86f7-wcgw4_de626df4-dde7-4bf7-b78b-1a6fdd7ab763/console/0.log" May 27 21:33:10.290882 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:33:10.290781 2564 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f77bb86f7-wcgw4" May 27 21:33:10.340721 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:33:10.340624 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/de626df4-dde7-4bf7-b78b-1a6fdd7ab763-oauth-serving-cert\") pod \"de626df4-dde7-4bf7-b78b-1a6fdd7ab763\" (UID: \"de626df4-dde7-4bf7-b78b-1a6fdd7ab763\") " May 27 21:33:10.340721 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:33:10.340703 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/de626df4-dde7-4bf7-b78b-1a6fdd7ab763-console-config\") pod \"de626df4-dde7-4bf7-b78b-1a6fdd7ab763\" (UID: \"de626df4-dde7-4bf7-b78b-1a6fdd7ab763\") " May 27 21:33:10.340962 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:33:10.340729 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de626df4-dde7-4bf7-b78b-1a6fdd7ab763-trusted-ca-bundle\") pod \"de626df4-dde7-4bf7-b78b-1a6fdd7ab763\" (UID: \"de626df4-dde7-4bf7-b78b-1a6fdd7ab763\") " May 27 21:33:10.340962 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:33:10.340784 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s852n\" (UniqueName: \"kubernetes.io/projected/de626df4-dde7-4bf7-b78b-1a6fdd7ab763-kube-api-access-s852n\") pod \"de626df4-dde7-4bf7-b78b-1a6fdd7ab763\" (UID: \"de626df4-dde7-4bf7-b78b-1a6fdd7ab763\") " May 27 21:33:10.340962 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:33:10.340808 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/de626df4-dde7-4bf7-b78b-1a6fdd7ab763-service-ca\") pod \"de626df4-dde7-4bf7-b78b-1a6fdd7ab763\" (UID: \"de626df4-dde7-4bf7-b78b-1a6fdd7ab763\") " May 27 21:33:10.340962 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:33:10.340863 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/de626df4-dde7-4bf7-b78b-1a6fdd7ab763-console-oauth-config\") pod \"de626df4-dde7-4bf7-b78b-1a6fdd7ab763\" (UID: \"de626df4-dde7-4bf7-b78b-1a6fdd7ab763\") " May 27 21:33:10.340962 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:33:10.340888 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/de626df4-dde7-4bf7-b78b-1a6fdd7ab763-console-serving-cert\") pod \"de626df4-dde7-4bf7-b78b-1a6fdd7ab763\" (UID: \"de626df4-dde7-4bf7-b78b-1a6fdd7ab763\") " May 27 21:33:10.341210 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:33:10.341177 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de626df4-dde7-4bf7-b78b-1a6fdd7ab763-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "de626df4-dde7-4bf7-b78b-1a6fdd7ab763" (UID: "de626df4-dde7-4bf7-b78b-1a6fdd7ab763"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 27 21:33:10.341283 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:33:10.341218 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de626df4-dde7-4bf7-b78b-1a6fdd7ab763-service-ca" (OuterVolumeSpecName: "service-ca") pod "de626df4-dde7-4bf7-b78b-1a6fdd7ab763" (UID: "de626df4-dde7-4bf7-b78b-1a6fdd7ab763"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 27 21:33:10.341283 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:33:10.341212 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de626df4-dde7-4bf7-b78b-1a6fdd7ab763-console-config" (OuterVolumeSpecName: "console-config") pod "de626df4-dde7-4bf7-b78b-1a6fdd7ab763" (UID: "de626df4-dde7-4bf7-b78b-1a6fdd7ab763"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 27 21:33:10.341283 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:33:10.341241 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de626df4-dde7-4bf7-b78b-1a6fdd7ab763-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "de626df4-dde7-4bf7-b78b-1a6fdd7ab763" (UID: "de626df4-dde7-4bf7-b78b-1a6fdd7ab763"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 27 21:33:10.343066 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:33:10.343026 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de626df4-dde7-4bf7-b78b-1a6fdd7ab763-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "de626df4-dde7-4bf7-b78b-1a6fdd7ab763" (UID: "de626df4-dde7-4bf7-b78b-1a6fdd7ab763"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 27 21:33:10.343165 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:33:10.343067 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de626df4-dde7-4bf7-b78b-1a6fdd7ab763-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "de626df4-dde7-4bf7-b78b-1a6fdd7ab763" (UID: "de626df4-dde7-4bf7-b78b-1a6fdd7ab763"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 27 21:33:10.343165 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:33:10.343074 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de626df4-dde7-4bf7-b78b-1a6fdd7ab763-kube-api-access-s852n" (OuterVolumeSpecName: "kube-api-access-s852n") pod "de626df4-dde7-4bf7-b78b-1a6fdd7ab763" (UID: "de626df4-dde7-4bf7-b78b-1a6fdd7ab763"). InnerVolumeSpecName "kube-api-access-s852n". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 27 21:33:10.441685 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:33:10.441634 2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/de626df4-dde7-4bf7-b78b-1a6fdd7ab763-console-oauth-config\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:33:10.441685 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:33:10.441677 2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/de626df4-dde7-4bf7-b78b-1a6fdd7ab763-console-serving-cert\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:33:10.441685 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:33:10.441688 2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/de626df4-dde7-4bf7-b78b-1a6fdd7ab763-oauth-serving-cert\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:33:10.441685 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:33:10.441698 2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/de626df4-dde7-4bf7-b78b-1a6fdd7ab763-console-config\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:33:10.441685 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:33:10.441706 2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de626df4-dde7-4bf7-b78b-1a6fdd7ab763-trusted-ca-bundle\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:33:10.441971 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:33:10.441717 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s852n\" (UniqueName: \"kubernetes.io/projected/de626df4-dde7-4bf7-b78b-1a6fdd7ab763-kube-api-access-s852n\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:33:10.441971 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:33:10.441727 2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/de626df4-dde7-4bf7-b78b-1a6fdd7ab763-service-ca\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:33:11.044489 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:33:11.044459 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6f77bb86f7-wcgw4_de626df4-dde7-4bf7-b78b-1a6fdd7ab763/console/0.log" May 27 21:33:11.044694 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:33:11.044505 2564 generic.go:358] "Generic (PLEG): container finished" podID="de626df4-dde7-4bf7-b78b-1a6fdd7ab763" containerID="043161d01ef6c663f9605bf562a48ee144cc18e85a9458aa8408805504e6bcae" exitCode=2 May 27 21:33:11.044694 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:33:11.044542 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f77bb86f7-wcgw4" event={"ID":"de626df4-dde7-4bf7-b78b-1a6fdd7ab763","Type":"ContainerDied","Data":"043161d01ef6c663f9605bf562a48ee144cc18e85a9458aa8408805504e6bcae"} May 27 21:33:11.044694 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:33:11.044562 2564 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f77bb86f7-wcgw4" May 27 21:33:11.044694 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:33:11.044583 2564 scope.go:117] "RemoveContainer" containerID="043161d01ef6c663f9605bf562a48ee144cc18e85a9458aa8408805504e6bcae" May 27 21:33:11.044694 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:33:11.044569 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f77bb86f7-wcgw4" event={"ID":"de626df4-dde7-4bf7-b78b-1a6fdd7ab763","Type":"ContainerDied","Data":"544c4a777237919c2b5418e1800935b601bb0689fc8c8b56186478e44cd40a16"} May 27 21:33:11.052902 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:33:11.052883 2564 scope.go:117] "RemoveContainer" containerID="043161d01ef6c663f9605bf562a48ee144cc18e85a9458aa8408805504e6bcae" May 27 21:33:11.053166 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:33:11.053147 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"043161d01ef6c663f9605bf562a48ee144cc18e85a9458aa8408805504e6bcae\": container with ID starting with 043161d01ef6c663f9605bf562a48ee144cc18e85a9458aa8408805504e6bcae not found: ID does not exist" containerID="043161d01ef6c663f9605bf562a48ee144cc18e85a9458aa8408805504e6bcae" May 27 21:33:11.053440 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:33:11.053181 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"043161d01ef6c663f9605bf562a48ee144cc18e85a9458aa8408805504e6bcae"} err="failed to get container status \"043161d01ef6c663f9605bf562a48ee144cc18e85a9458aa8408805504e6bcae\": rpc error: code = NotFound desc = could not find container \"043161d01ef6c663f9605bf562a48ee144cc18e85a9458aa8408805504e6bcae\": container with ID starting with 043161d01ef6c663f9605bf562a48ee144cc18e85a9458aa8408805504e6bcae not found: ID does not exist" May 27 21:33:11.065592 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:33:11.065561 2564 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f77bb86f7-wcgw4"] May 27 21:33:11.069367 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:33:11.069345 2564 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6f77bb86f7-wcgw4"] May 27 21:33:11.084046 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:33:11.084014 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de626df4-dde7-4bf7-b78b-1a6fdd7ab763" path="/var/lib/kubelet/pods/de626df4-dde7-4bf7-b78b-1a6fdd7ab763/volumes" May 27 21:33:46.965517 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:33:46.965487 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-d9gcd_16b8cf1e-1181-43fc-92e0-635b8523248d/console-operator/2.log" May 27 21:33:46.970232 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:33:46.970203 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-d9gcd_16b8cf1e-1181-43fc-92e0-635b8523248d/console-operator/2.log" May 27 21:33:46.974119 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:33:46.974095 2564 kubelet.go:1628] "Image garbage collection succeeded" May 27 21:34:09.823950 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:09.823915 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-77fdcbc69d-wvxxh"] May 27 21:34:09.825388 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:09.824207 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de626df4-dde7-4bf7-b78b-1a6fdd7ab763" containerName="console" May 27 21:34:09.825388 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:09.824218 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="de626df4-dde7-4bf7-b78b-1a6fdd7ab763" containerName="console" May 27 21:34:09.825388 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:09.824287 2564 memory_manager.go:356] "RemoveStaleState removing state" podUID="de626df4-dde7-4bf7-b78b-1a6fdd7ab763" containerName="console" May 27 21:34:09.826026 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:09.826006 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77fdcbc69d-wvxxh" May 27 21:34:09.838250 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:09.838220 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77fdcbc69d-wvxxh"] May 27 21:34:09.914567 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:09.914528 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b520c3be-d3f3-4cb2-a739-444a8a5975ba-service-ca\") pod \"console-77fdcbc69d-wvxxh\" (UID: \"b520c3be-d3f3-4cb2-a739-444a8a5975ba\") " pod="openshift-console/console-77fdcbc69d-wvxxh" May 27 21:34:09.914567 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:09.914567 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b520c3be-d3f3-4cb2-a739-444a8a5975ba-console-serving-cert\") pod \"console-77fdcbc69d-wvxxh\" (UID: \"b520c3be-d3f3-4cb2-a739-444a8a5975ba\") " pod="openshift-console/console-77fdcbc69d-wvxxh" May 27 21:34:09.914820 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:09.914590 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b520c3be-d3f3-4cb2-a739-444a8a5975ba-console-config\") pod \"console-77fdcbc69d-wvxxh\" (UID: \"b520c3be-d3f3-4cb2-a739-444a8a5975ba\") " pod="openshift-console/console-77fdcbc69d-wvxxh" May 27 21:34:09.914820 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:09.914720 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkxnp\" (UniqueName: \"kubernetes.io/projected/b520c3be-d3f3-4cb2-a739-444a8a5975ba-kube-api-access-tkxnp\") pod \"console-77fdcbc69d-wvxxh\" (UID: \"b520c3be-d3f3-4cb2-a739-444a8a5975ba\") " pod="openshift-console/console-77fdcbc69d-wvxxh" May 27 21:34:09.914820 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:09.914764 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b520c3be-d3f3-4cb2-a739-444a8a5975ba-trusted-ca-bundle\") pod \"console-77fdcbc69d-wvxxh\" (UID: \"b520c3be-d3f3-4cb2-a739-444a8a5975ba\") " pod="openshift-console/console-77fdcbc69d-wvxxh" May 27 21:34:09.914820 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:09.914812 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b520c3be-d3f3-4cb2-a739-444a8a5975ba-oauth-serving-cert\") pod \"console-77fdcbc69d-wvxxh\" (UID: \"b520c3be-d3f3-4cb2-a739-444a8a5975ba\") " pod="openshift-console/console-77fdcbc69d-wvxxh" May 27 21:34:09.914972 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:09.914838 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b520c3be-d3f3-4cb2-a739-444a8a5975ba-console-oauth-config\") pod \"console-77fdcbc69d-wvxxh\" (UID: \"b520c3be-d3f3-4cb2-a739-444a8a5975ba\") " pod="openshift-console/console-77fdcbc69d-wvxxh" May 27 21:34:10.016100 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:10.016059 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tkxnp\" (UniqueName: \"kubernetes.io/projected/b520c3be-d3f3-4cb2-a739-444a8a5975ba-kube-api-access-tkxnp\") pod \"console-77fdcbc69d-wvxxh\" (UID: \"b520c3be-d3f3-4cb2-a739-444a8a5975ba\") " pod="openshift-console/console-77fdcbc69d-wvxxh" May 27 21:34:10.016100 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:10.016099 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b520c3be-d3f3-4cb2-a739-444a8a5975ba-trusted-ca-bundle\") pod \"console-77fdcbc69d-wvxxh\" (UID: \"b520c3be-d3f3-4cb2-a739-444a8a5975ba\") " pod="openshift-console/console-77fdcbc69d-wvxxh" May 27 21:34:10.016409 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:10.016124 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b520c3be-d3f3-4cb2-a739-444a8a5975ba-oauth-serving-cert\") pod \"console-77fdcbc69d-wvxxh\" (UID: \"b520c3be-d3f3-4cb2-a739-444a8a5975ba\") " pod="openshift-console/console-77fdcbc69d-wvxxh" May 27 21:34:10.016409 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:10.016146 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b520c3be-d3f3-4cb2-a739-444a8a5975ba-console-oauth-config\") pod \"console-77fdcbc69d-wvxxh\" (UID: \"b520c3be-d3f3-4cb2-a739-444a8a5975ba\") " pod="openshift-console/console-77fdcbc69d-wvxxh" May 27 21:34:10.016409 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:10.016206 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b520c3be-d3f3-4cb2-a739-444a8a5975ba-service-ca\") pod \"console-77fdcbc69d-wvxxh\" (UID: \"b520c3be-d3f3-4cb2-a739-444a8a5975ba\") " pod="openshift-console/console-77fdcbc69d-wvxxh" May 27 21:34:10.016409 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:10.016240 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b520c3be-d3f3-4cb2-a739-444a8a5975ba-console-serving-cert\") pod \"console-77fdcbc69d-wvxxh\" (UID: \"b520c3be-d3f3-4cb2-a739-444a8a5975ba\") " pod="openshift-console/console-77fdcbc69d-wvxxh" May 27 21:34:10.016409 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:10.016294 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b520c3be-d3f3-4cb2-a739-444a8a5975ba-console-config\") pod \"console-77fdcbc69d-wvxxh\" (UID: \"b520c3be-d3f3-4cb2-a739-444a8a5975ba\") " pod="openshift-console/console-77fdcbc69d-wvxxh" May 27 21:34:10.016935 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:10.016907 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b520c3be-d3f3-4cb2-a739-444a8a5975ba-oauth-serving-cert\") pod \"console-77fdcbc69d-wvxxh\" (UID: \"b520c3be-d3f3-4cb2-a739-444a8a5975ba\") " pod="openshift-console/console-77fdcbc69d-wvxxh" May 27 21:34:10.017094 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:10.017074 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b520c3be-d3f3-4cb2-a739-444a8a5975ba-trusted-ca-bundle\") pod \"console-77fdcbc69d-wvxxh\" (UID: \"b520c3be-d3f3-4cb2-a739-444a8a5975ba\") " pod="openshift-console/console-77fdcbc69d-wvxxh" May 27 21:34:10.017163 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:10.017016 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b520c3be-d3f3-4cb2-a739-444a8a5975ba-console-config\") pod \"console-77fdcbc69d-wvxxh\" (UID: \"b520c3be-d3f3-4cb2-a739-444a8a5975ba\") " pod="openshift-console/console-77fdcbc69d-wvxxh" May 27 21:34:10.017163 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:10.016992 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b520c3be-d3f3-4cb2-a739-444a8a5975ba-service-ca\") pod \"console-77fdcbc69d-wvxxh\" (UID: \"b520c3be-d3f3-4cb2-a739-444a8a5975ba\") " pod="openshift-console/console-77fdcbc69d-wvxxh" May 27 21:34:10.019447 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:10.019406 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b520c3be-d3f3-4cb2-a739-444a8a5975ba-console-oauth-config\") pod \"console-77fdcbc69d-wvxxh\" (UID: \"b520c3be-d3f3-4cb2-a739-444a8a5975ba\") " pod="openshift-console/console-77fdcbc69d-wvxxh" May 27 21:34:10.019542 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:10.019411 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b520c3be-d3f3-4cb2-a739-444a8a5975ba-console-serving-cert\") pod \"console-77fdcbc69d-wvxxh\" (UID: \"b520c3be-d3f3-4cb2-a739-444a8a5975ba\") " pod="openshift-console/console-77fdcbc69d-wvxxh" May 27 21:34:10.025108 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:10.025089 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkxnp\" (UniqueName: \"kubernetes.io/projected/b520c3be-d3f3-4cb2-a739-444a8a5975ba-kube-api-access-tkxnp\") pod \"console-77fdcbc69d-wvxxh\" (UID: \"b520c3be-d3f3-4cb2-a739-444a8a5975ba\") " pod="openshift-console/console-77fdcbc69d-wvxxh" May 27 21:34:10.137735 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:10.137633 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77fdcbc69d-wvxxh" May 27 21:34:10.276339 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:10.276313 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77fdcbc69d-wvxxh"] May 27 21:34:10.278902 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:34:10.278860 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb520c3be_d3f3_4cb2_a739_444a8a5975ba.slice/crio-94dfa220ba9c462016503c7d300379ea47e1a1e8fc661e0ede9e599ee6cec65e WatchSource:0}: Error finding container 94dfa220ba9c462016503c7d300379ea47e1a1e8fc661e0ede9e599ee6cec65e: Status 404 returned error can't find the container with id 94dfa220ba9c462016503c7d300379ea47e1a1e8fc661e0ede9e599ee6cec65e May 27 21:34:10.280610 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:10.280592 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 27 21:34:11.223025 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:11.222985 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77fdcbc69d-wvxxh" event={"ID":"b520c3be-d3f3-4cb2-a739-444a8a5975ba","Type":"ContainerStarted","Data":"09a6651f670b9534a33330c162420ab38910952b6d70f720428d38680f1df663"} May 27 21:34:11.223444 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:11.223033 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77fdcbc69d-wvxxh" event={"ID":"b520c3be-d3f3-4cb2-a739-444a8a5975ba","Type":"ContainerStarted","Data":"94dfa220ba9c462016503c7d300379ea47e1a1e8fc661e0ede9e599ee6cec65e"} May 27 21:34:11.242559 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:11.242498 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-77fdcbc69d-wvxxh" podStartSLOduration=2.242476979 podStartE2EDuration="2.242476979s" podCreationTimestamp="2026-05-27 21:34:09 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-27 21:34:11.241320918 +0000 UTC m=+324.715525649" watchObservedRunningTime="2026-05-27 21:34:11.242476979 +0000 UTC m=+324.716681708" May 27 21:34:18.256432 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:18.256396 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-vm9vc"] May 27 21:34:18.258730 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:18.258708 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vm9vc" May 27 21:34:18.261619 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:18.261600 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" May 27 21:34:18.269453 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:18.269429 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vm9vc"] May 27 21:34:18.284926 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:18.284901 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2174f1a7-3c35-461b-a60d-b2b6714e35f8-original-pull-secret\") pod \"global-pull-secret-syncer-vm9vc\" (UID: \"2174f1a7-3c35-461b-a60d-b2b6714e35f8\") " pod="kube-system/global-pull-secret-syncer-vm9vc" May 27 21:34:18.285016 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:18.284962 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2174f1a7-3c35-461b-a60d-b2b6714e35f8-dbus\") pod \"global-pull-secret-syncer-vm9vc\" (UID: \"2174f1a7-3c35-461b-a60d-b2b6714e35f8\") " pod="kube-system/global-pull-secret-syncer-vm9vc" May 27 21:34:18.285016 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:18.284999 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2174f1a7-3c35-461b-a60d-b2b6714e35f8-kubelet-config\") pod \"global-pull-secret-syncer-vm9vc\" (UID: \"2174f1a7-3c35-461b-a60d-b2b6714e35f8\") " pod="kube-system/global-pull-secret-syncer-vm9vc" May 27 21:34:18.386173 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:18.386147 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2174f1a7-3c35-461b-a60d-b2b6714e35f8-dbus\") pod \"global-pull-secret-syncer-vm9vc\" (UID: \"2174f1a7-3c35-461b-a60d-b2b6714e35f8\") " pod="kube-system/global-pull-secret-syncer-vm9vc" May 27 21:34:18.386350 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:18.386204 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2174f1a7-3c35-461b-a60d-b2b6714e35f8-kubelet-config\") pod \"global-pull-secret-syncer-vm9vc\" (UID: \"2174f1a7-3c35-461b-a60d-b2b6714e35f8\") " pod="kube-system/global-pull-secret-syncer-vm9vc" May 27 21:34:18.386350 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:18.386294 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2174f1a7-3c35-461b-a60d-b2b6714e35f8-original-pull-secret\") pod \"global-pull-secret-syncer-vm9vc\" (UID: \"2174f1a7-3c35-461b-a60d-b2b6714e35f8\") " pod="kube-system/global-pull-secret-syncer-vm9vc" May 27 21:34:18.386466 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:18.386344 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2174f1a7-3c35-461b-a60d-b2b6714e35f8-kubelet-config\") pod \"global-pull-secret-syncer-vm9vc\" (UID: \"2174f1a7-3c35-461b-a60d-b2b6714e35f8\") " pod="kube-system/global-pull-secret-syncer-vm9vc" May 27 21:34:18.386466 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:18.386364 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2174f1a7-3c35-461b-a60d-b2b6714e35f8-dbus\") pod \"global-pull-secret-syncer-vm9vc\" (UID: \"2174f1a7-3c35-461b-a60d-b2b6714e35f8\") " pod="kube-system/global-pull-secret-syncer-vm9vc" May 27 21:34:18.388565 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:18.388545 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2174f1a7-3c35-461b-a60d-b2b6714e35f8-original-pull-secret\") pod \"global-pull-secret-syncer-vm9vc\" (UID: \"2174f1a7-3c35-461b-a60d-b2b6714e35f8\") " pod="kube-system/global-pull-secret-syncer-vm9vc" May 27 21:34:18.568581 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:18.568497 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vm9vc" May 27 21:34:18.692590 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:18.692554 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vm9vc"] May 27 21:34:18.695316 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:34:18.695288 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2174f1a7_3c35_461b_a60d_b2b6714e35f8.slice/crio-205afc4421855fcda7ebd0eb2331fb9d3e0bdcfad00d32e4964d236aa63cd477 WatchSource:0}: Error finding container 205afc4421855fcda7ebd0eb2331fb9d3e0bdcfad00d32e4964d236aa63cd477: Status 404 returned error can't find the container with id 205afc4421855fcda7ebd0eb2331fb9d3e0bdcfad00d32e4964d236aa63cd477 May 27 21:34:19.246292 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:19.246245 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vm9vc" event={"ID":"2174f1a7-3c35-461b-a60d-b2b6714e35f8","Type":"ContainerStarted","Data":"205afc4421855fcda7ebd0eb2331fb9d3e0bdcfad00d32e4964d236aa63cd477"} May 27 21:34:20.137783 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:20.137743 2564 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-77fdcbc69d-wvxxh" May 27 21:34:20.137783 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:20.137783 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-77fdcbc69d-wvxxh" May 27 21:34:20.144567 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:20.144543 2564 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-77fdcbc69d-wvxxh" May 27 21:34:20.253528 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:20.253497 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-77fdcbc69d-wvxxh" May 27 21:34:20.300418 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:20.300381 2564 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-69c9b4fcb4-jmngr"] May 27 21:34:23.259836 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:23.259803 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vm9vc" event={"ID":"2174f1a7-3c35-461b-a60d-b2b6714e35f8","Type":"ContainerStarted","Data":"d79544d3265ebdd6a5d46ec4a8e62ac14c5ae57eb2ad57b0d26cb4c03805915e"} May 27 21:34:23.277849 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:23.277772 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-vm9vc" podStartSLOduration=0.836509614 podStartE2EDuration="5.277728695s" podCreationTimestamp="2026-05-27 21:34:18 +0000 UTC" firstStartedPulling="2026-05-27 21:34:18.696857983 +0000 UTC m=+332.171062690" lastFinishedPulling="2026-05-27 21:34:23.13807706 +0000 UTC m=+336.612281771" observedRunningTime="2026-05-27 21:34:23.276781366 +0000 UTC m=+336.750986109" watchObservedRunningTime="2026-05-27 21:34:23.277728695 +0000 UTC m=+336.751933426" May 27 21:34:45.321576 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:45.321462 2564 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-69c9b4fcb4-jmngr" podUID="963c876d-fac5-46e6-90b4-63ba0cb87714" containerName="console" containerID="cri-o://fa659a29bdeb10644f68d7a1375aeaac1bf58f5f04b708d79738f33e1beeda67" gracePeriod=15 May 27 21:34:45.560471 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:45.560444 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-69c9b4fcb4-jmngr_963c876d-fac5-46e6-90b4-63ba0cb87714/console/0.log" May 27 21:34:45.560618 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:45.560520 2564 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69c9b4fcb4-jmngr" May 27 21:34:45.606743 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:45.606653 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/963c876d-fac5-46e6-90b4-63ba0cb87714-console-config\") pod \"963c876d-fac5-46e6-90b4-63ba0cb87714\" (UID: \"963c876d-fac5-46e6-90b4-63ba0cb87714\") " May 27 21:34:45.606743 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:45.606694 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/963c876d-fac5-46e6-90b4-63ba0cb87714-trusted-ca-bundle\") pod \"963c876d-fac5-46e6-90b4-63ba0cb87714\" (UID: \"963c876d-fac5-46e6-90b4-63ba0cb87714\") " May 27 21:34:45.606743 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:45.606731 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/963c876d-fac5-46e6-90b4-63ba0cb87714-oauth-serving-cert\") pod \"963c876d-fac5-46e6-90b4-63ba0cb87714\" (UID: \"963c876d-fac5-46e6-90b4-63ba0cb87714\") " May 27 21:34:45.606985 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:45.606758 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ffrf7\" (UniqueName: \"kubernetes.io/projected/963c876d-fac5-46e6-90b4-63ba0cb87714-kube-api-access-ffrf7\") pod \"963c876d-fac5-46e6-90b4-63ba0cb87714\" (UID: \"963c876d-fac5-46e6-90b4-63ba0cb87714\") " May 27 21:34:45.606985 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:45.606816 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/963c876d-fac5-46e6-90b4-63ba0cb87714-console-oauth-config\") pod \"963c876d-fac5-46e6-90b4-63ba0cb87714\" (UID: \"963c876d-fac5-46e6-90b4-63ba0cb87714\") " May 27 21:34:45.606985 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:45.606851 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/963c876d-fac5-46e6-90b4-63ba0cb87714-console-serving-cert\") pod \"963c876d-fac5-46e6-90b4-63ba0cb87714\" (UID: \"963c876d-fac5-46e6-90b4-63ba0cb87714\") " May 27 21:34:45.606985 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:45.606893 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/963c876d-fac5-46e6-90b4-63ba0cb87714-service-ca\") pod \"963c876d-fac5-46e6-90b4-63ba0cb87714\" (UID: \"963c876d-fac5-46e6-90b4-63ba0cb87714\") " May 27 21:34:45.607186 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:45.607055 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/963c876d-fac5-46e6-90b4-63ba0cb87714-console-config" (OuterVolumeSpecName: "console-config") pod "963c876d-fac5-46e6-90b4-63ba0cb87714" (UID: "963c876d-fac5-46e6-90b4-63ba0cb87714"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 27 21:34:45.607186 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:45.607145 2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/963c876d-fac5-46e6-90b4-63ba0cb87714-console-config\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:34:45.607317 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:45.607212 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/963c876d-fac5-46e6-90b4-63ba0cb87714-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "963c876d-fac5-46e6-90b4-63ba0cb87714" (UID: "963c876d-fac5-46e6-90b4-63ba0cb87714"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 27 21:34:45.607317 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:45.607304 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/963c876d-fac5-46e6-90b4-63ba0cb87714-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "963c876d-fac5-46e6-90b4-63ba0cb87714" (UID: "963c876d-fac5-46e6-90b4-63ba0cb87714"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 27 21:34:45.607442 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:45.607417 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/963c876d-fac5-46e6-90b4-63ba0cb87714-service-ca" (OuterVolumeSpecName: "service-ca") pod "963c876d-fac5-46e6-90b4-63ba0cb87714" (UID: "963c876d-fac5-46e6-90b4-63ba0cb87714"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 27 21:34:45.609209 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:45.609182 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/963c876d-fac5-46e6-90b4-63ba0cb87714-kube-api-access-ffrf7" (OuterVolumeSpecName: "kube-api-access-ffrf7") pod "963c876d-fac5-46e6-90b4-63ba0cb87714" (UID: "963c876d-fac5-46e6-90b4-63ba0cb87714"). InnerVolumeSpecName "kube-api-access-ffrf7". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 27 21:34:45.609209 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:45.609192 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/963c876d-fac5-46e6-90b4-63ba0cb87714-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "963c876d-fac5-46e6-90b4-63ba0cb87714" (UID: "963c876d-fac5-46e6-90b4-63ba0cb87714"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 27 21:34:45.609375 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:45.609288 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/963c876d-fac5-46e6-90b4-63ba0cb87714-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "963c876d-fac5-46e6-90b4-63ba0cb87714" (UID: "963c876d-fac5-46e6-90b4-63ba0cb87714"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 27 21:34:45.708545 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:45.708502 2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/963c876d-fac5-46e6-90b4-63ba0cb87714-trusted-ca-bundle\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:34:45.708545 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:45.708535 2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/963c876d-fac5-46e6-90b4-63ba0cb87714-oauth-serving-cert\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:34:45.708545 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:45.708547 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ffrf7\" (UniqueName: \"kubernetes.io/projected/963c876d-fac5-46e6-90b4-63ba0cb87714-kube-api-access-ffrf7\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:34:45.708545 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:45.708556 2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/963c876d-fac5-46e6-90b4-63ba0cb87714-console-oauth-config\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:34:45.708800 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:45.708565 2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/963c876d-fac5-46e6-90b4-63ba0cb87714-console-serving-cert\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:34:45.708800 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:45.708576 2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/963c876d-fac5-46e6-90b4-63ba0cb87714-service-ca\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:34:46.327704 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:46.327670 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-69c9b4fcb4-jmngr_963c876d-fac5-46e6-90b4-63ba0cb87714/console/0.log" May 27 21:34:46.328084 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:46.327717 2564 generic.go:358] "Generic (PLEG): container finished" podID="963c876d-fac5-46e6-90b4-63ba0cb87714" containerID="fa659a29bdeb10644f68d7a1375aeaac1bf58f5f04b708d79738f33e1beeda67" exitCode=2 May 27 21:34:46.328084 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:46.327760 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69c9b4fcb4-jmngr" event={"ID":"963c876d-fac5-46e6-90b4-63ba0cb87714","Type":"ContainerDied","Data":"fa659a29bdeb10644f68d7a1375aeaac1bf58f5f04b708d79738f33e1beeda67"} May 27 21:34:46.328084 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:46.327792 2564 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69c9b4fcb4-jmngr" May 27 21:34:46.328084 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:46.327822 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69c9b4fcb4-jmngr" event={"ID":"963c876d-fac5-46e6-90b4-63ba0cb87714","Type":"ContainerDied","Data":"777449a0bed9903ffc2327a763984e4a5de4fee071559fb267ab85c18e6649d7"} May 27 21:34:46.328084 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:46.327841 2564 scope.go:117] "RemoveContainer" containerID="fa659a29bdeb10644f68d7a1375aeaac1bf58f5f04b708d79738f33e1beeda67" May 27 21:34:46.336902 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:46.336881 2564 scope.go:117] "RemoveContainer" containerID="fa659a29bdeb10644f68d7a1375aeaac1bf58f5f04b708d79738f33e1beeda67" May 27 21:34:46.337211 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:34:46.337195 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa659a29bdeb10644f68d7a1375aeaac1bf58f5f04b708d79738f33e1beeda67\": container with ID starting with fa659a29bdeb10644f68d7a1375aeaac1bf58f5f04b708d79738f33e1beeda67 not found: ID does not exist" containerID="fa659a29bdeb10644f68d7a1375aeaac1bf58f5f04b708d79738f33e1beeda67" May 27 21:34:46.337260 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:46.337219 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa659a29bdeb10644f68d7a1375aeaac1bf58f5f04b708d79738f33e1beeda67"} err="failed to get container status \"fa659a29bdeb10644f68d7a1375aeaac1bf58f5f04b708d79738f33e1beeda67\": rpc error: code = NotFound desc = could not find container \"fa659a29bdeb10644f68d7a1375aeaac1bf58f5f04b708d79738f33e1beeda67\": container with ID starting with fa659a29bdeb10644f68d7a1375aeaac1bf58f5f04b708d79738f33e1beeda67 not found: ID does not exist" May 27 21:34:46.350940 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:46.350905 2564 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-69c9b4fcb4-jmngr"] May 27 21:34:46.357574 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:46.357545 2564 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-69c9b4fcb4-jmngr"] May 27 21:34:47.084362 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:34:47.084321 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="963c876d-fac5-46e6-90b4-63ba0cb87714" path="/var/lib/kubelet/pods/963c876d-fac5-46e6-90b4-63ba0cb87714/volumes" May 27 21:35:49.762717 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:35:49.762672 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d6tfcs"] May 27 21:35:49.763250 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:35:49.763135 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="963c876d-fac5-46e6-90b4-63ba0cb87714" containerName="console" May 27 21:35:49.763250 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:35:49.763154 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="963c876d-fac5-46e6-90b4-63ba0cb87714" containerName="console" May 27 21:35:49.763417 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:35:49.763260 2564 memory_manager.go:356] "RemoveStaleState removing state" podUID="963c876d-fac5-46e6-90b4-63ba0cb87714" containerName="console" May 27 21:35:49.766318 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:35:49.766299 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d6tfcs" May 27 21:35:49.768856 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:35:49.768830 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" May 27 21:35:49.768984 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:35:49.768962 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" May 27 21:35:49.769928 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:35:49.769913 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-tmqch\"" May 27 21:35:49.774786 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:35:49.774762 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d6tfcs"] May 27 21:35:49.832687 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:35:49.832649 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a964662a-ebdb-4946-bbc8-09df056e8ce1-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d6tfcs\" (UID: \"a964662a-ebdb-4946-bbc8-09df056e8ce1\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d6tfcs" May 27 21:35:49.832883 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:35:49.832758 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzf6c\" (UniqueName: \"kubernetes.io/projected/a964662a-ebdb-4946-bbc8-09df056e8ce1-kube-api-access-kzf6c\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d6tfcs\" (UID: \"a964662a-ebdb-4946-bbc8-09df056e8ce1\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d6tfcs" May 27 21:35:49.832883 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:35:49.832827 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a964662a-ebdb-4946-bbc8-09df056e8ce1-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d6tfcs\" (UID: \"a964662a-ebdb-4946-bbc8-09df056e8ce1\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d6tfcs" May 27 21:35:49.933631 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:35:49.933594 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a964662a-ebdb-4946-bbc8-09df056e8ce1-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d6tfcs\" (UID: \"a964662a-ebdb-4946-bbc8-09df056e8ce1\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d6tfcs" May 27 21:35:49.933788 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:35:49.933748 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a964662a-ebdb-4946-bbc8-09df056e8ce1-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d6tfcs\" (UID: \"a964662a-ebdb-4946-bbc8-09df056e8ce1\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d6tfcs" May 27 21:35:49.933854 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:35:49.933800 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kzf6c\" (UniqueName: \"kubernetes.io/projected/a964662a-ebdb-4946-bbc8-09df056e8ce1-kube-api-access-kzf6c\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d6tfcs\" (UID: \"a964662a-ebdb-4946-bbc8-09df056e8ce1\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d6tfcs" May 27 21:35:49.934006 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:35:49.933984 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a964662a-ebdb-4946-bbc8-09df056e8ce1-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d6tfcs\" (UID: \"a964662a-ebdb-4946-bbc8-09df056e8ce1\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d6tfcs" May 27 21:35:49.934100 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:35:49.934079 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a964662a-ebdb-4946-bbc8-09df056e8ce1-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d6tfcs\" (UID: \"a964662a-ebdb-4946-bbc8-09df056e8ce1\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d6tfcs" May 27 21:35:49.943861 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:35:49.943841 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzf6c\" (UniqueName: \"kubernetes.io/projected/a964662a-ebdb-4946-bbc8-09df056e8ce1-kube-api-access-kzf6c\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d6tfcs\" (UID: \"a964662a-ebdb-4946-bbc8-09df056e8ce1\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d6tfcs" May 27 21:35:50.076159 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:35:50.076059 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d6tfcs" May 27 21:35:50.201955 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:35:50.201895 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d6tfcs"] May 27 21:35:50.204700 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:35:50.204669 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda964662a_ebdb_4946_bbc8_09df056e8ce1.slice/crio-4dcea2ae5f5ad3fc2bd7afa7e90e32d7ca76c7edcb5a2a0408e374df155ed195 WatchSource:0}: Error finding container 4dcea2ae5f5ad3fc2bd7afa7e90e32d7ca76c7edcb5a2a0408e374df155ed195: Status 404 returned error can't find the container with id 4dcea2ae5f5ad3fc2bd7afa7e90e32d7ca76c7edcb5a2a0408e374df155ed195 May 27 21:35:50.514956 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:35:50.514914 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d6tfcs" event={"ID":"a964662a-ebdb-4946-bbc8-09df056e8ce1","Type":"ContainerStarted","Data":"4dcea2ae5f5ad3fc2bd7afa7e90e32d7ca76c7edcb5a2a0408e374df155ed195"} May 27 21:35:56.541846 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:35:56.541741 2564 generic.go:358] "Generic (PLEG): container finished" podID="a964662a-ebdb-4946-bbc8-09df056e8ce1" containerID="c5eee0b71bb6d828a0edc45facc8ec93cff89e933bdce01c1b44ac3b47ae5fc6" exitCode=0 May 27 21:35:56.541846 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:35:56.541788 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d6tfcs" event={"ID":"a964662a-ebdb-4946-bbc8-09df056e8ce1","Type":"ContainerDied","Data":"c5eee0b71bb6d828a0edc45facc8ec93cff89e933bdce01c1b44ac3b47ae5fc6"} May 27 21:35:59.551921 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:35:59.551881 2564 generic.go:358] "Generic (PLEG): container finished" podID="a964662a-ebdb-4946-bbc8-09df056e8ce1" containerID="15ad938c8f75f957926dc0db94e16868e7ea4e74dd0fb0dedcc89000051c9387" exitCode=0 May 27 21:35:59.552315 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:35:59.551956 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d6tfcs" event={"ID":"a964662a-ebdb-4946-bbc8-09df056e8ce1","Type":"ContainerDied","Data":"15ad938c8f75f957926dc0db94e16868e7ea4e74dd0fb0dedcc89000051c9387"} May 27 21:36:05.572602 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:05.572560 2564 generic.go:358] "Generic (PLEG): container finished" podID="a964662a-ebdb-4946-bbc8-09df056e8ce1" containerID="be0e8103f338594c722900063d3d751a663b68f9c61cc29f57176d783411af00" exitCode=0 May 27 21:36:05.572962 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:05.572613 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d6tfcs" event={"ID":"a964662a-ebdb-4946-bbc8-09df056e8ce1","Type":"ContainerDied","Data":"be0e8103f338594c722900063d3d751a663b68f9c61cc29f57176d783411af00"} May 27 21:36:06.703340 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:06.703316 2564 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d6tfcs" May 27 21:36:06.783290 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:06.783225 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a964662a-ebdb-4946-bbc8-09df056e8ce1-bundle\") pod \"a964662a-ebdb-4946-bbc8-09df056e8ce1\" (UID: \"a964662a-ebdb-4946-bbc8-09df056e8ce1\") " May 27 21:36:06.783474 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:06.783367 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kzf6c\" (UniqueName: \"kubernetes.io/projected/a964662a-ebdb-4946-bbc8-09df056e8ce1-kube-api-access-kzf6c\") pod \"a964662a-ebdb-4946-bbc8-09df056e8ce1\" (UID: \"a964662a-ebdb-4946-bbc8-09df056e8ce1\") " May 27 21:36:06.783474 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:06.783393 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a964662a-ebdb-4946-bbc8-09df056e8ce1-util\") pod \"a964662a-ebdb-4946-bbc8-09df056e8ce1\" (UID: \"a964662a-ebdb-4946-bbc8-09df056e8ce1\") " May 27 21:36:06.783985 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:06.783959 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a964662a-ebdb-4946-bbc8-09df056e8ce1-bundle" (OuterVolumeSpecName: "bundle") pod "a964662a-ebdb-4946-bbc8-09df056e8ce1" (UID: "a964662a-ebdb-4946-bbc8-09df056e8ce1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 27 21:36:06.785636 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:06.785613 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a964662a-ebdb-4946-bbc8-09df056e8ce1-kube-api-access-kzf6c" (OuterVolumeSpecName: "kube-api-access-kzf6c") pod "a964662a-ebdb-4946-bbc8-09df056e8ce1" (UID: "a964662a-ebdb-4946-bbc8-09df056e8ce1"). InnerVolumeSpecName "kube-api-access-kzf6c". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 27 21:36:06.787552 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:06.787528 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a964662a-ebdb-4946-bbc8-09df056e8ce1-util" (OuterVolumeSpecName: "util") pod "a964662a-ebdb-4946-bbc8-09df056e8ce1" (UID: "a964662a-ebdb-4946-bbc8-09df056e8ce1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 27 21:36:06.884604 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:06.884521 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a964662a-ebdb-4946-bbc8-09df056e8ce1-bundle\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:36:06.884604 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:06.884555 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kzf6c\" (UniqueName: \"kubernetes.io/projected/a964662a-ebdb-4946-bbc8-09df056e8ce1-kube-api-access-kzf6c\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:36:06.884604 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:06.884566 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a964662a-ebdb-4946-bbc8-09df056e8ce1-util\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:36:07.581535 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:07.581497 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d6tfcs" event={"ID":"a964662a-ebdb-4946-bbc8-09df056e8ce1","Type":"ContainerDied","Data":"4dcea2ae5f5ad3fc2bd7afa7e90e32d7ca76c7edcb5a2a0408e374df155ed195"} May 27 21:36:07.581535 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:07.581535 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4dcea2ae5f5ad3fc2bd7afa7e90e32d7ca76c7edcb5a2a0408e374df155ed195" May 27 21:36:07.581535 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:07.581534 2564 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d6tfcs" May 27 21:36:12.928386 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:12.928352 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-scp94"] May 27 21:36:12.928732 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:12.928638 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a964662a-ebdb-4946-bbc8-09df056e8ce1" containerName="util" May 27 21:36:12.928732 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:12.928652 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a964662a-ebdb-4946-bbc8-09df056e8ce1" containerName="util" May 27 21:36:12.928732 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:12.928664 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a964662a-ebdb-4946-bbc8-09df056e8ce1" containerName="pull" May 27 21:36:12.928732 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:12.928671 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a964662a-ebdb-4946-bbc8-09df056e8ce1" containerName="pull" May 27 21:36:12.928732 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:12.928686 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a964662a-ebdb-4946-bbc8-09df056e8ce1" containerName="extract" May 27 21:36:12.928732 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:12.928691 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a964662a-ebdb-4946-bbc8-09df056e8ce1" containerName="extract" May 27 21:36:12.928908 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:12.928741 2564 memory_manager.go:356] "RemoveStaleState removing state" podUID="a964662a-ebdb-4946-bbc8-09df056e8ce1" containerName="extract" May 27 21:36:12.932739 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:12.932720 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-scp94" May 27 21:36:12.935923 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:12.935898 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-msxh4\"" May 27 21:36:12.936222 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:12.936208 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" May 27 21:36:12.936547 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:12.936533 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" May 27 21:36:12.941293 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:12.941255 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/64a25826-25b3-46b7-9faa-d334559b1912-tmp\") pod \"cert-manager-operator-controller-manager-54b9655956-scp94\" (UID: \"64a25826-25b3-46b7-9faa-d334559b1912\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-scp94" May 27 21:36:12.941395 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:12.941324 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6zlx\" (UniqueName: \"kubernetes.io/projected/64a25826-25b3-46b7-9faa-d334559b1912-kube-api-access-z6zlx\") pod \"cert-manager-operator-controller-manager-54b9655956-scp94\" (UID: \"64a25826-25b3-46b7-9faa-d334559b1912\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-scp94" May 27 21:36:12.944033 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:12.944012 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-scp94"] May 27 21:36:13.041774 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:13.041739 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/64a25826-25b3-46b7-9faa-d334559b1912-tmp\") pod \"cert-manager-operator-controller-manager-54b9655956-scp94\" (UID: \"64a25826-25b3-46b7-9faa-d334559b1912\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-scp94" May 27 21:36:13.041774 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:13.041776 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-z6zlx\" (UniqueName: \"kubernetes.io/projected/64a25826-25b3-46b7-9faa-d334559b1912-kube-api-access-z6zlx\") pod \"cert-manager-operator-controller-manager-54b9655956-scp94\" (UID: \"64a25826-25b3-46b7-9faa-d334559b1912\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-scp94" May 27 21:36:13.042148 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:13.042129 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/64a25826-25b3-46b7-9faa-d334559b1912-tmp\") pod \"cert-manager-operator-controller-manager-54b9655956-scp94\" (UID: \"64a25826-25b3-46b7-9faa-d334559b1912\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-scp94" May 27 21:36:13.050949 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:13.050931 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6zlx\" (UniqueName: \"kubernetes.io/projected/64a25826-25b3-46b7-9faa-d334559b1912-kube-api-access-z6zlx\") pod \"cert-manager-operator-controller-manager-54b9655956-scp94\" (UID: \"64a25826-25b3-46b7-9faa-d334559b1912\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-scp94" May 27 21:36:13.241971 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:13.241880 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-scp94" May 27 21:36:13.376729 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:13.376697 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-scp94"] May 27 21:36:13.380377 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:36:13.380346 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64a25826_25b3_46b7_9faa_d334559b1912.slice/crio-a87018adac73e1040fa3cb8123231831191489fcc942de78193475bed512f390 WatchSource:0}: Error finding container a87018adac73e1040fa3cb8123231831191489fcc942de78193475bed512f390: Status 404 returned error can't find the container with id a87018adac73e1040fa3cb8123231831191489fcc942de78193475bed512f390 May 27 21:36:13.601669 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:13.601586 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-scp94" event={"ID":"64a25826-25b3-46b7-9faa-d334559b1912","Type":"ContainerStarted","Data":"a87018adac73e1040fa3cb8123231831191489fcc942de78193475bed512f390"} May 27 21:36:15.608872 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:15.608840 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-scp94" event={"ID":"64a25826-25b3-46b7-9faa-d334559b1912","Type":"ContainerStarted","Data":"b4c7b0fcc20075a0856ba9442e46934e141129e90e7dfb99a4c31356b7a82006"} May 27 21:36:15.638787 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:15.638739 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-scp94" podStartSLOduration=1.861210393 podStartE2EDuration="3.638724735s" podCreationTimestamp="2026-05-27 21:36:12 +0000 UTC" firstStartedPulling="2026-05-27 21:36:13.382553448 +0000 UTC m=+446.856758155" lastFinishedPulling="2026-05-27 21:36:15.160067789 +0000 UTC m=+448.634272497" observedRunningTime="2026-05-27 21:36:15.637076042 +0000 UTC m=+449.111280758" watchObservedRunningTime="2026-05-27 21:36:15.638724735 +0000 UTC m=+449.112929462" May 27 21:36:17.420865 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:17.420830 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4s6d8"] May 27 21:36:17.424313 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:17.424296 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4s6d8" May 27 21:36:17.428581 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:17.428562 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" May 27 21:36:17.429589 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:17.429572 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-tmqch\"" May 27 21:36:17.429699 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:17.429573 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" May 27 21:36:17.446441 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:17.446417 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4s6d8"] May 27 21:36:17.471081 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:17.471057 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b35b29cd-2df0-4c39-bf0d-c39659fd2372-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4s6d8\" (UID: \"b35b29cd-2df0-4c39-bf0d-c39659fd2372\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4s6d8" May 27 21:36:17.471193 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:17.471098 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b35b29cd-2df0-4c39-bf0d-c39659fd2372-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4s6d8\" (UID: \"b35b29cd-2df0-4c39-bf0d-c39659fd2372\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4s6d8" May 27 21:36:17.471193 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:17.471134 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqcmv\" (UniqueName: \"kubernetes.io/projected/b35b29cd-2df0-4c39-bf0d-c39659fd2372-kube-api-access-dqcmv\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4s6d8\" (UID: \"b35b29cd-2df0-4c39-bf0d-c39659fd2372\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4s6d8" May 27 21:36:17.571494 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:17.571463 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b35b29cd-2df0-4c39-bf0d-c39659fd2372-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4s6d8\" (UID: \"b35b29cd-2df0-4c39-bf0d-c39659fd2372\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4s6d8" May 27 21:36:17.571643 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:17.571514 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dqcmv\" (UniqueName: \"kubernetes.io/projected/b35b29cd-2df0-4c39-bf0d-c39659fd2372-kube-api-access-dqcmv\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4s6d8\" (UID: \"b35b29cd-2df0-4c39-bf0d-c39659fd2372\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4s6d8" May 27 21:36:17.571643 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:17.571562 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b35b29cd-2df0-4c39-bf0d-c39659fd2372-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4s6d8\" (UID: \"b35b29cd-2df0-4c39-bf0d-c39659fd2372\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4s6d8" May 27 21:36:17.571870 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:17.571851 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b35b29cd-2df0-4c39-bf0d-c39659fd2372-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4s6d8\" (UID: \"b35b29cd-2df0-4c39-bf0d-c39659fd2372\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4s6d8" May 27 21:36:17.571910 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:17.571858 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b35b29cd-2df0-4c39-bf0d-c39659fd2372-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4s6d8\" (UID: \"b35b29cd-2df0-4c39-bf0d-c39659fd2372\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4s6d8" May 27 21:36:17.583382 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:17.583360 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqcmv\" (UniqueName: \"kubernetes.io/projected/b35b29cd-2df0-4c39-bf0d-c39659fd2372-kube-api-access-dqcmv\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4s6d8\" (UID: \"b35b29cd-2df0-4c39-bf0d-c39659fd2372\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4s6d8" May 27 21:36:17.732810 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:17.732729 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4s6d8" May 27 21:36:17.859184 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:17.859148 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4s6d8"] May 27 21:36:17.862218 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:36:17.862187 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb35b29cd_2df0_4c39_bf0d_c39659fd2372.slice/crio-ed1ddb32d607d08283adf617abfc10283789a5422f49c16b42c5d76b876f4ae6 WatchSource:0}: Error finding container ed1ddb32d607d08283adf617abfc10283789a5422f49c16b42c5d76b876f4ae6: Status 404 returned error can't find the container with id ed1ddb32d607d08283adf617abfc10283789a5422f49c16b42c5d76b876f4ae6 May 27 21:36:18.619341 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:18.619308 2564 generic.go:358] "Generic (PLEG): container finished" podID="b35b29cd-2df0-4c39-bf0d-c39659fd2372" containerID="7f6f2e20cf2be942b98c0983c7ff83e73d043866c404419d6a03399a10831940" exitCode=0 May 27 21:36:18.619660 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:18.619354 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4s6d8" event={"ID":"b35b29cd-2df0-4c39-bf0d-c39659fd2372","Type":"ContainerDied","Data":"7f6f2e20cf2be942b98c0983c7ff83e73d043866c404419d6a03399a10831940"} May 27 21:36:18.619660 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:18.619376 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4s6d8" event={"ID":"b35b29cd-2df0-4c39-bf0d-c39659fd2372","Type":"ContainerStarted","Data":"ed1ddb32d607d08283adf617abfc10283789a5422f49c16b42c5d76b876f4ae6"} May 27 21:36:19.974472 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:19.974435 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-68b757865b-7dhms"] May 27 21:36:19.978001 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:19.977978 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-68b757865b-7dhms" May 27 21:36:19.981462 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:19.981438 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" May 27 21:36:19.981576 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:19.981495 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-gl6j2\"" May 27 21:36:19.982485 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:19.982463 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" May 27 21:36:19.988850 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:19.988827 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7988\" (UniqueName: \"kubernetes.io/projected/3c899b9d-11c6-4c08-aa48-c2de383d9906-kube-api-access-x7988\") pod \"cert-manager-cainjector-68b757865b-7dhms\" (UID: \"3c899b9d-11c6-4c08-aa48-c2de383d9906\") " pod="cert-manager/cert-manager-cainjector-68b757865b-7dhms" May 27 21:36:19.988975 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:19.988869 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3c899b9d-11c6-4c08-aa48-c2de383d9906-bound-sa-token\") pod \"cert-manager-cainjector-68b757865b-7dhms\" (UID: \"3c899b9d-11c6-4c08-aa48-c2de383d9906\") " pod="cert-manager/cert-manager-cainjector-68b757865b-7dhms" May 27 21:36:19.995108 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:19.995079 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-68b757865b-7dhms"] May 27 21:36:20.089956 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:20.089920 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-x7988\" (UniqueName: \"kubernetes.io/projected/3c899b9d-11c6-4c08-aa48-c2de383d9906-kube-api-access-x7988\") pod \"cert-manager-cainjector-68b757865b-7dhms\" (UID: \"3c899b9d-11c6-4c08-aa48-c2de383d9906\") " pod="cert-manager/cert-manager-cainjector-68b757865b-7dhms" May 27 21:36:20.090122 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:20.089975 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3c899b9d-11c6-4c08-aa48-c2de383d9906-bound-sa-token\") pod \"cert-manager-cainjector-68b757865b-7dhms\" (UID: \"3c899b9d-11c6-4c08-aa48-c2de383d9906\") " pod="cert-manager/cert-manager-cainjector-68b757865b-7dhms" May 27 21:36:20.102894 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:20.102862 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3c899b9d-11c6-4c08-aa48-c2de383d9906-bound-sa-token\") pod \"cert-manager-cainjector-68b757865b-7dhms\" (UID: \"3c899b9d-11c6-4c08-aa48-c2de383d9906\") " pod="cert-manager/cert-manager-cainjector-68b757865b-7dhms" May 27 21:36:20.103684 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:20.103640 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7988\" (UniqueName: \"kubernetes.io/projected/3c899b9d-11c6-4c08-aa48-c2de383d9906-kube-api-access-x7988\") pod \"cert-manager-cainjector-68b757865b-7dhms\" (UID: \"3c899b9d-11c6-4c08-aa48-c2de383d9906\") " pod="cert-manager/cert-manager-cainjector-68b757865b-7dhms" May 27 21:36:20.294510 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:20.294423 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-68b757865b-7dhms" May 27 21:36:20.949497 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:20.949476 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-68b757865b-7dhms"] May 27 21:36:20.952060 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:36:20.952035 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3c899b9d_11c6_4c08_aa48_c2de383d9906.slice/crio-134b2c5a254bc67a16d8371b90d30896a89a8bedee930727f28143a5dea0d924 WatchSource:0}: Error finding container 134b2c5a254bc67a16d8371b90d30896a89a8bedee930727f28143a5dea0d924: Status 404 returned error can't find the container with id 134b2c5a254bc67a16d8371b90d30896a89a8bedee930727f28143a5dea0d924 May 27 21:36:21.630391 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:21.630347 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-68b757865b-7dhms" event={"ID":"3c899b9d-11c6-4c08-aa48-c2de383d9906","Type":"ContainerStarted","Data":"134b2c5a254bc67a16d8371b90d30896a89a8bedee930727f28143a5dea0d924"} May 27 21:36:21.631942 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:21.631914 2564 generic.go:358] "Generic (PLEG): container finished" podID="b35b29cd-2df0-4c39-bf0d-c39659fd2372" containerID="019252619da28b0821215e2a64715a915d9926a74bcb3d0a222a09296910a2fb" exitCode=0 May 27 21:36:21.632095 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:21.631952 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4s6d8" event={"ID":"b35b29cd-2df0-4c39-bf0d-c39659fd2372","Type":"ContainerDied","Data":"019252619da28b0821215e2a64715a915d9926a74bcb3d0a222a09296910a2fb"} May 27 21:36:22.637961 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:22.637929 2564 generic.go:358] "Generic (PLEG): container finished" podID="b35b29cd-2df0-4c39-bf0d-c39659fd2372" containerID="93ab87a942e1a2b40455c2cc2330e7ffa2379c4f94db217e476072a2b02d6c4e" exitCode=0 May 27 21:36:22.638350 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:22.637982 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4s6d8" event={"ID":"b35b29cd-2df0-4c39-bf0d-c39659fd2372","Type":"ContainerDied","Data":"93ab87a942e1a2b40455c2cc2330e7ffa2379c4f94db217e476072a2b02d6c4e"} May 27 21:36:23.641820 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:23.641779 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-68b757865b-7dhms" event={"ID":"3c899b9d-11c6-4c08-aa48-c2de383d9906","Type":"ContainerStarted","Data":"0eb88a96cfd334cef1d42811ebee091b26a2b1e2dd8af5cdfa9cb2fdee86f468"} May 27 21:36:23.658682 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:23.658627 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-68b757865b-7dhms" podStartSLOduration=2.105934579 podStartE2EDuration="4.658613324s" podCreationTimestamp="2026-05-27 21:36:19 +0000 UTC" firstStartedPulling="2026-05-27 21:36:20.953820318 +0000 UTC m=+454.428025030" lastFinishedPulling="2026-05-27 21:36:23.506499066 +0000 UTC m=+456.980703775" observedRunningTime="2026-05-27 21:36:23.657562232 +0000 UTC m=+457.131766996" watchObservedRunningTime="2026-05-27 21:36:23.658613324 +0000 UTC m=+457.132818054" May 27 21:36:23.771930 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:23.771900 2564 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4s6d8" May 27 21:36:23.814889 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:23.814856 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dqcmv\" (UniqueName: \"kubernetes.io/projected/b35b29cd-2df0-4c39-bf0d-c39659fd2372-kube-api-access-dqcmv\") pod \"b35b29cd-2df0-4c39-bf0d-c39659fd2372\" (UID: \"b35b29cd-2df0-4c39-bf0d-c39659fd2372\") " May 27 21:36:23.815056 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:23.814907 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b35b29cd-2df0-4c39-bf0d-c39659fd2372-bundle\") pod \"b35b29cd-2df0-4c39-bf0d-c39659fd2372\" (UID: \"b35b29cd-2df0-4c39-bf0d-c39659fd2372\") " May 27 21:36:23.815260 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:23.815233 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b35b29cd-2df0-4c39-bf0d-c39659fd2372-bundle" (OuterVolumeSpecName: "bundle") pod "b35b29cd-2df0-4c39-bf0d-c39659fd2372" (UID: "b35b29cd-2df0-4c39-bf0d-c39659fd2372"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 27 21:36:23.816870 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:23.816843 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b35b29cd-2df0-4c39-bf0d-c39659fd2372-kube-api-access-dqcmv" (OuterVolumeSpecName: "kube-api-access-dqcmv") pod "b35b29cd-2df0-4c39-bf0d-c39659fd2372" (UID: "b35b29cd-2df0-4c39-bf0d-c39659fd2372"). InnerVolumeSpecName "kube-api-access-dqcmv". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 27 21:36:23.915838 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:23.915806 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b35b29cd-2df0-4c39-bf0d-c39659fd2372-util\") pod \"b35b29cd-2df0-4c39-bf0d-c39659fd2372\" (UID: \"b35b29cd-2df0-4c39-bf0d-c39659fd2372\") " May 27 21:36:23.915996 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:23.915972 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dqcmv\" (UniqueName: \"kubernetes.io/projected/b35b29cd-2df0-4c39-bf0d-c39659fd2372-kube-api-access-dqcmv\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:36:23.915996 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:23.915984 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b35b29cd-2df0-4c39-bf0d-c39659fd2372-bundle\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:36:23.920703 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:23.920663 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b35b29cd-2df0-4c39-bf0d-c39659fd2372-util" (OuterVolumeSpecName: "util") pod "b35b29cd-2df0-4c39-bf0d-c39659fd2372" (UID: "b35b29cd-2df0-4c39-bf0d-c39659fd2372"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 27 21:36:24.016941 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:24.016861 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b35b29cd-2df0-4c39-bf0d-c39659fd2372-util\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:36:24.647085 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:24.647048 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4s6d8" event={"ID":"b35b29cd-2df0-4c39-bf0d-c39659fd2372","Type":"ContainerDied","Data":"ed1ddb32d607d08283adf617abfc10283789a5422f49c16b42c5d76b876f4ae6"} May 27 21:36:24.647085 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:24.647086 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ed1ddb32d607d08283adf617abfc10283789a5422f49c16b42c5d76b876f4ae6" May 27 21:36:24.647524 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:24.647087 2564 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4s6d8" May 27 21:36:29.240744 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:29.240707 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-79c8d999ff-xkt7q"] May 27 21:36:29.241176 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:29.240995 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b35b29cd-2df0-4c39-bf0d-c39659fd2372" containerName="pull" May 27 21:36:29.241176 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:29.241007 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b35b29cd-2df0-4c39-bf0d-c39659fd2372" containerName="pull" May 27 21:36:29.241176 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:29.241015 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b35b29cd-2df0-4c39-bf0d-c39659fd2372" containerName="extract" May 27 21:36:29.241176 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:29.241020 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b35b29cd-2df0-4c39-bf0d-c39659fd2372" containerName="extract" May 27 21:36:29.241176 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:29.241038 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b35b29cd-2df0-4c39-bf0d-c39659fd2372" containerName="util" May 27 21:36:29.241176 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:29.241044 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b35b29cd-2df0-4c39-bf0d-c39659fd2372" containerName="util" May 27 21:36:29.241176 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:29.241092 2564 memory_manager.go:356] "RemoveStaleState removing state" podUID="b35b29cd-2df0-4c39-bf0d-c39659fd2372" containerName="extract" May 27 21:36:29.285856 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:29.285813 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-79c8d999ff-xkt7q"] May 27 21:36:29.286008 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:29.285944 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-79c8d999ff-xkt7q" May 27 21:36:29.288997 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:29.288977 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-q6z8r\"" May 27 21:36:29.358496 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:29.358460 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kz6c\" (UniqueName: \"kubernetes.io/projected/bfea8c30-daf6-4af4-9186-240e225f81fb-kube-api-access-5kz6c\") pod \"cert-manager-79c8d999ff-xkt7q\" (UID: \"bfea8c30-daf6-4af4-9186-240e225f81fb\") " pod="cert-manager/cert-manager-79c8d999ff-xkt7q" May 27 21:36:29.358671 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:29.358509 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bfea8c30-daf6-4af4-9186-240e225f81fb-bound-sa-token\") pod \"cert-manager-79c8d999ff-xkt7q\" (UID: \"bfea8c30-daf6-4af4-9186-240e225f81fb\") " pod="cert-manager/cert-manager-79c8d999ff-xkt7q" May 27 21:36:29.458988 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:29.458948 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5kz6c\" (UniqueName: \"kubernetes.io/projected/bfea8c30-daf6-4af4-9186-240e225f81fb-kube-api-access-5kz6c\") pod \"cert-manager-79c8d999ff-xkt7q\" (UID: \"bfea8c30-daf6-4af4-9186-240e225f81fb\") " pod="cert-manager/cert-manager-79c8d999ff-xkt7q" May 27 21:36:29.459135 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:29.458999 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bfea8c30-daf6-4af4-9186-240e225f81fb-bound-sa-token\") pod \"cert-manager-79c8d999ff-xkt7q\" (UID: \"bfea8c30-daf6-4af4-9186-240e225f81fb\") " pod="cert-manager/cert-manager-79c8d999ff-xkt7q" May 27 21:36:29.483281 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:29.483240 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bfea8c30-daf6-4af4-9186-240e225f81fb-bound-sa-token\") pod \"cert-manager-79c8d999ff-xkt7q\" (UID: \"bfea8c30-daf6-4af4-9186-240e225f81fb\") " pod="cert-manager/cert-manager-79c8d999ff-xkt7q" May 27 21:36:29.483424 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:29.483325 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kz6c\" (UniqueName: \"kubernetes.io/projected/bfea8c30-daf6-4af4-9186-240e225f81fb-kube-api-access-5kz6c\") pod \"cert-manager-79c8d999ff-xkt7q\" (UID: \"bfea8c30-daf6-4af4-9186-240e225f81fb\") " pod="cert-manager/cert-manager-79c8d999ff-xkt7q" May 27 21:36:29.595414 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:29.595332 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-79c8d999ff-xkt7q" May 27 21:36:29.727505 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:29.727471 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-79c8d999ff-xkt7q"] May 27 21:36:29.730319 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:36:29.730288 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbfea8c30_daf6_4af4_9186_240e225f81fb.slice/crio-89184700e66ed0616c91b318c18df210f8e6e3c12ac5ee972c494d551c5586bd WatchSource:0}: Error finding container 89184700e66ed0616c91b318c18df210f8e6e3c12ac5ee972c494d551c5586bd: Status 404 returned error can't find the container with id 89184700e66ed0616c91b318c18df210f8e6e3c12ac5ee972c494d551c5586bd May 27 21:36:30.667127 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:30.667092 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-79c8d999ff-xkt7q" event={"ID":"bfea8c30-daf6-4af4-9186-240e225f81fb","Type":"ContainerStarted","Data":"ee61718b7762f003e1ea5fc381e3d5a545c3a3f14d78e78eb1670f89f1c68b85"} May 27 21:36:30.667127 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:30.667131 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-79c8d999ff-xkt7q" event={"ID":"bfea8c30-daf6-4af4-9186-240e225f81fb","Type":"ContainerStarted","Data":"89184700e66ed0616c91b318c18df210f8e6e3c12ac5ee972c494d551c5586bd"} May 27 21:36:30.686939 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:30.686882 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager/cert-manager-79c8d999ff-xkt7q" podStartSLOduration=1.686864789 podStartE2EDuration="1.686864789s" podCreationTimestamp="2026-05-27 21:36:29 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-27 21:36:30.685763846 +0000 UTC m=+464.159968576" watchObservedRunningTime="2026-05-27 21:36:30.686864789 +0000 UTC m=+464.161069519" May 27 21:36:42.581639 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:42.581600 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5ntlkk"] May 27 21:36:42.584037 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:42.584014 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5ntlkk" May 27 21:36:42.586617 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:42.586595 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" May 27 21:36:42.586701 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:42.586595 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" May 27 21:36:42.588250 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:42.588235 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-tmqch\"" May 27 21:36:42.593955 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:42.593932 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5ntlkk"] May 27 21:36:42.665166 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:42.665132 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3fc476c6-45f7-453d-a2e0-b0e969088902-bundle\") pod \"3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5ntlkk\" (UID: \"3fc476c6-45f7-453d-a2e0-b0e969088902\") " pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5ntlkk" May 27 21:36:42.665166 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:42.665167 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rmjq\" (UniqueName: \"kubernetes.io/projected/3fc476c6-45f7-453d-a2e0-b0e969088902-kube-api-access-5rmjq\") pod \"3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5ntlkk\" (UID: \"3fc476c6-45f7-453d-a2e0-b0e969088902\") " pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5ntlkk" May 27 21:36:42.665401 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:42.665202 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3fc476c6-45f7-453d-a2e0-b0e969088902-util\") pod \"3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5ntlkk\" (UID: \"3fc476c6-45f7-453d-a2e0-b0e969088902\") " pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5ntlkk" May 27 21:36:42.766359 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:42.766320 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3fc476c6-45f7-453d-a2e0-b0e969088902-bundle\") pod \"3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5ntlkk\" (UID: \"3fc476c6-45f7-453d-a2e0-b0e969088902\") " pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5ntlkk" May 27 21:36:42.766359 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:42.766361 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5rmjq\" (UniqueName: \"kubernetes.io/projected/3fc476c6-45f7-453d-a2e0-b0e969088902-kube-api-access-5rmjq\") pod \"3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5ntlkk\" (UID: \"3fc476c6-45f7-453d-a2e0-b0e969088902\") " pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5ntlkk" May 27 21:36:42.766545 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:42.766398 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3fc476c6-45f7-453d-a2e0-b0e969088902-util\") pod \"3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5ntlkk\" (UID: \"3fc476c6-45f7-453d-a2e0-b0e969088902\") " pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5ntlkk" May 27 21:36:42.766770 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:42.766747 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3fc476c6-45f7-453d-a2e0-b0e969088902-bundle\") pod \"3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5ntlkk\" (UID: \"3fc476c6-45f7-453d-a2e0-b0e969088902\") " pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5ntlkk" May 27 21:36:42.766807 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:42.766783 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3fc476c6-45f7-453d-a2e0-b0e969088902-util\") pod \"3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5ntlkk\" (UID: \"3fc476c6-45f7-453d-a2e0-b0e969088902\") " pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5ntlkk" May 27 21:36:42.775347 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:42.775325 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rmjq\" (UniqueName: \"kubernetes.io/projected/3fc476c6-45f7-453d-a2e0-b0e969088902-kube-api-access-5rmjq\") pod \"3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5ntlkk\" (UID: \"3fc476c6-45f7-453d-a2e0-b0e969088902\") " pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5ntlkk" May 27 21:36:42.894491 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:42.894407 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5ntlkk" May 27 21:36:43.039015 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:43.038993 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5ntlkk"] May 27 21:36:43.040849 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:36:43.040818 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fc476c6_45f7_453d_a2e0_b0e969088902.slice/crio-b3515b57f922d6403aeca9cda69ec41e65c1354701a0bb8f9174f26fc0f2e43c WatchSource:0}: Error finding container b3515b57f922d6403aeca9cda69ec41e65c1354701a0bb8f9174f26fc0f2e43c: Status 404 returned error can't find the container with id b3515b57f922d6403aeca9cda69ec41e65c1354701a0bb8f9174f26fc0f2e43c May 27 21:36:43.713685 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:43.713647 2564 generic.go:358] "Generic (PLEG): container finished" podID="3fc476c6-45f7-453d-a2e0-b0e969088902" containerID="fe1474e886c4aa070a017e103d177ca4b6a64ac267bc0036bd8a9457dc232529" exitCode=0 May 27 21:36:43.714046 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:43.713737 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5ntlkk" event={"ID":"3fc476c6-45f7-453d-a2e0-b0e969088902","Type":"ContainerDied","Data":"fe1474e886c4aa070a017e103d177ca4b6a64ac267bc0036bd8a9457dc232529"} May 27 21:36:43.714046 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:43.713781 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5ntlkk" event={"ID":"3fc476c6-45f7-453d-a2e0-b0e969088902","Type":"ContainerStarted","Data":"b3515b57f922d6403aeca9cda69ec41e65c1354701a0bb8f9174f26fc0f2e43c"} May 27 21:36:44.718675 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:44.718642 2564 generic.go:358] "Generic (PLEG): container finished" podID="3fc476c6-45f7-453d-a2e0-b0e969088902" containerID="8c22646ebff1a05ccfa995f28d536743a4d2a6daa7d61bea33624a181d2a4e80" exitCode=0 May 27 21:36:44.719044 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:44.718709 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5ntlkk" event={"ID":"3fc476c6-45f7-453d-a2e0-b0e969088902","Type":"ContainerDied","Data":"8c22646ebff1a05ccfa995f28d536743a4d2a6daa7d61bea33624a181d2a4e80"} May 27 21:36:45.724032 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:45.723997 2564 generic.go:358] "Generic (PLEG): container finished" podID="3fc476c6-45f7-453d-a2e0-b0e969088902" containerID="aa0da7ff381ea607b39e15822cce7514d38c7a5f6f05edd010a7721a0ac739ef" exitCode=0 May 27 21:36:45.724420 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:45.724111 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5ntlkk" event={"ID":"3fc476c6-45f7-453d-a2e0-b0e969088902","Type":"ContainerDied","Data":"aa0da7ff381ea607b39e15822cce7514d38c7a5f6f05edd010a7721a0ac739ef"} May 27 21:36:46.846793 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:46.846770 2564 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5ntlkk" May 27 21:36:46.900584 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:46.900557 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3fc476c6-45f7-453d-a2e0-b0e969088902-bundle\") pod \"3fc476c6-45f7-453d-a2e0-b0e969088902\" (UID: \"3fc476c6-45f7-453d-a2e0-b0e969088902\") " May 27 21:36:46.900701 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:46.900612 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3fc476c6-45f7-453d-a2e0-b0e969088902-util\") pod \"3fc476c6-45f7-453d-a2e0-b0e969088902\" (UID: \"3fc476c6-45f7-453d-a2e0-b0e969088902\") " May 27 21:36:46.900701 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:46.900689 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5rmjq\" (UniqueName: \"kubernetes.io/projected/3fc476c6-45f7-453d-a2e0-b0e969088902-kube-api-access-5rmjq\") pod \"3fc476c6-45f7-453d-a2e0-b0e969088902\" (UID: \"3fc476c6-45f7-453d-a2e0-b0e969088902\") " May 27 21:36:46.901238 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:46.901216 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fc476c6-45f7-453d-a2e0-b0e969088902-bundle" (OuterVolumeSpecName: "bundle") pod "3fc476c6-45f7-453d-a2e0-b0e969088902" (UID: "3fc476c6-45f7-453d-a2e0-b0e969088902"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 27 21:36:46.902707 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:46.902685 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fc476c6-45f7-453d-a2e0-b0e969088902-kube-api-access-5rmjq" (OuterVolumeSpecName: "kube-api-access-5rmjq") pod "3fc476c6-45f7-453d-a2e0-b0e969088902" (UID: "3fc476c6-45f7-453d-a2e0-b0e969088902"). InnerVolumeSpecName "kube-api-access-5rmjq". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 27 21:36:46.906032 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:46.906000 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fc476c6-45f7-453d-a2e0-b0e969088902-util" (OuterVolumeSpecName: "util") pod "3fc476c6-45f7-453d-a2e0-b0e969088902" (UID: "3fc476c6-45f7-453d-a2e0-b0e969088902"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 27 21:36:47.001953 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:47.001886 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3fc476c6-45f7-453d-a2e0-b0e969088902-util\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:36:47.001953 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:47.001910 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5rmjq\" (UniqueName: \"kubernetes.io/projected/3fc476c6-45f7-453d-a2e0-b0e969088902-kube-api-access-5rmjq\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:36:47.001953 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:47.001921 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3fc476c6-45f7-453d-a2e0-b0e969088902-bundle\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:36:47.731407 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:47.731378 2564 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5ntlkk" May 27 21:36:47.731407 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:47.731385 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5ntlkk" event={"ID":"3fc476c6-45f7-453d-a2e0-b0e969088902","Type":"ContainerDied","Data":"b3515b57f922d6403aeca9cda69ec41e65c1354701a0bb8f9174f26fc0f2e43c"} May 27 21:36:47.731614 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:47.731425 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b3515b57f922d6403aeca9cda69ec41e65c1354701a0bb8f9174f26fc0f2e43c" May 27 21:36:52.435126 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:52.435093 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c9bs8t5"] May 27 21:36:52.435601 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:52.435454 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fc476c6-45f7-453d-a2e0-b0e969088902" containerName="extract" May 27 21:36:52.435601 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:52.435468 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fc476c6-45f7-453d-a2e0-b0e969088902" containerName="extract" May 27 21:36:52.435601 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:52.435478 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fc476c6-45f7-453d-a2e0-b0e969088902" containerName="util" May 27 21:36:52.435601 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:52.435483 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fc476c6-45f7-453d-a2e0-b0e969088902" containerName="util" May 27 21:36:52.435601 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:52.435490 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fc476c6-45f7-453d-a2e0-b0e969088902" containerName="pull" May 27 21:36:52.435601 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:52.435497 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fc476c6-45f7-453d-a2e0-b0e969088902" containerName="pull" May 27 21:36:52.435601 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:52.435572 2564 memory_manager.go:356] "RemoveStaleState removing state" podUID="3fc476c6-45f7-453d-a2e0-b0e969088902" containerName="extract" May 27 21:36:52.439219 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:52.439200 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c9bs8t5" May 27 21:36:52.443298 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:52.443249 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-tmqch\"" May 27 21:36:52.443816 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:52.443797 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" May 27 21:36:52.444107 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:52.444088 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" May 27 21:36:52.449307 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:52.449279 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c9bs8t5"] May 27 21:36:52.547310 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:52.547244 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/88de7aeb-b93b-4f17-91f9-a0489a81edde-bundle\") pod \"f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c9bs8t5\" (UID: \"88de7aeb-b93b-4f17-91f9-a0489a81edde\") " pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c9bs8t5" May 27 21:36:52.547495 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:52.547360 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/88de7aeb-b93b-4f17-91f9-a0489a81edde-util\") pod \"f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c9bs8t5\" (UID: \"88de7aeb-b93b-4f17-91f9-a0489a81edde\") " pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c9bs8t5" May 27 21:36:52.547495 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:52.547416 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7pkv\" (UniqueName: \"kubernetes.io/projected/88de7aeb-b93b-4f17-91f9-a0489a81edde-kube-api-access-h7pkv\") pod \"f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c9bs8t5\" (UID: \"88de7aeb-b93b-4f17-91f9-a0489a81edde\") " pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c9bs8t5" May 27 21:36:52.648364 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:52.648332 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/88de7aeb-b93b-4f17-91f9-a0489a81edde-util\") pod \"f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c9bs8t5\" (UID: \"88de7aeb-b93b-4f17-91f9-a0489a81edde\") " pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c9bs8t5" May 27 21:36:52.648540 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:52.648381 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-h7pkv\" (UniqueName: \"kubernetes.io/projected/88de7aeb-b93b-4f17-91f9-a0489a81edde-kube-api-access-h7pkv\") pod \"f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c9bs8t5\" (UID: \"88de7aeb-b93b-4f17-91f9-a0489a81edde\") " pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c9bs8t5" May 27 21:36:52.648540 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:52.648433 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/88de7aeb-b93b-4f17-91f9-a0489a81edde-bundle\") pod \"f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c9bs8t5\" (UID: \"88de7aeb-b93b-4f17-91f9-a0489a81edde\") " pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c9bs8t5" May 27 21:36:52.648742 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:52.648722 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/88de7aeb-b93b-4f17-91f9-a0489a81edde-util\") pod \"f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c9bs8t5\" (UID: \"88de7aeb-b93b-4f17-91f9-a0489a81edde\") " pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c9bs8t5" May 27 21:36:52.648780 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:52.648747 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/88de7aeb-b93b-4f17-91f9-a0489a81edde-bundle\") pod \"f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c9bs8t5\" (UID: \"88de7aeb-b93b-4f17-91f9-a0489a81edde\") " pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c9bs8t5" May 27 21:36:52.661778 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:52.661754 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7pkv\" (UniqueName: \"kubernetes.io/projected/88de7aeb-b93b-4f17-91f9-a0489a81edde-kube-api-access-h7pkv\") pod \"f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c9bs8t5\" (UID: \"88de7aeb-b93b-4f17-91f9-a0489a81edde\") " pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c9bs8t5" May 27 21:36:52.748936 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:52.748849 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c9bs8t5" May 27 21:36:52.889451 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:52.889412 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c9bs8t5"] May 27 21:36:52.892316 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:36:52.892287 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88de7aeb_b93b_4f17_91f9_a0489a81edde.slice/crio-fadef1bbe2ab41a1134443a09bea1a8f7e3e6bbbf9ae4b6af6033c46ef7fc1c4 WatchSource:0}: Error finding container fadef1bbe2ab41a1134443a09bea1a8f7e3e6bbbf9ae4b6af6033c46ef7fc1c4: Status 404 returned error can't find the container with id fadef1bbe2ab41a1134443a09bea1a8f7e3e6bbbf9ae4b6af6033c46ef7fc1c4 May 27 21:36:53.752330 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:53.752295 2564 generic.go:358] "Generic (PLEG): container finished" podID="88de7aeb-b93b-4f17-91f9-a0489a81edde" containerID="51a2ec81ddfdba33eef2ec138fee377cabfc332c8207098728f670ca40950eca" exitCode=0 May 27 21:36:53.752706 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:53.752391 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c9bs8t5" event={"ID":"88de7aeb-b93b-4f17-91f9-a0489a81edde","Type":"ContainerDied","Data":"51a2ec81ddfdba33eef2ec138fee377cabfc332c8207098728f670ca40950eca"} May 27 21:36:53.752706 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:53.752427 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c9bs8t5" event={"ID":"88de7aeb-b93b-4f17-91f9-a0489a81edde","Type":"ContainerStarted","Data":"fadef1bbe2ab41a1134443a09bea1a8f7e3e6bbbf9ae4b6af6033c46ef7fc1c4"} May 27 21:36:54.744737 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:54.744656 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7d799877b9-bjwnb"] May 27 21:36:54.747866 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:54.747849 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7d799877b9-bjwnb" May 27 21:36:54.751769 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:54.751739 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" May 27 21:36:54.752680 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:54.752656 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" May 27 21:36:54.752680 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:54.752666 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" May 27 21:36:54.753084 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:54.752656 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" May 27 21:36:54.753084 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:54.753019 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-898d8\"" May 27 21:36:54.757887 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:54.757865 2564 generic.go:358] "Generic (PLEG): container finished" podID="88de7aeb-b93b-4f17-91f9-a0489a81edde" containerID="92a7a9cdff37d63415471c630a774d2f8da9b754273ce17de6b5dadf71b244c7" exitCode=0 May 27 21:36:54.757989 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:54.757900 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c9bs8t5" event={"ID":"88de7aeb-b93b-4f17-91f9-a0489a81edde","Type":"ContainerDied","Data":"92a7a9cdff37d63415471c630a774d2f8da9b754273ce17de6b5dadf71b244c7"} May 27 21:36:54.782018 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:54.781998 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7d799877b9-bjwnb"] May 27 21:36:54.867384 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:54.867283 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/db87c04f-749b-4491-8b16-7c072e2714f9-webhook-cert\") pod \"opendatahub-operator-controller-manager-7d799877b9-bjwnb\" (UID: \"db87c04f-749b-4491-8b16-7c072e2714f9\") " pod="opendatahub/opendatahub-operator-controller-manager-7d799877b9-bjwnb" May 27 21:36:54.867384 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:54.867351 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8q6pl\" (UniqueName: \"kubernetes.io/projected/db87c04f-749b-4491-8b16-7c072e2714f9-kube-api-access-8q6pl\") pod \"opendatahub-operator-controller-manager-7d799877b9-bjwnb\" (UID: \"db87c04f-749b-4491-8b16-7c072e2714f9\") " pod="opendatahub/opendatahub-operator-controller-manager-7d799877b9-bjwnb" May 27 21:36:54.867664 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:54.867432 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/db87c04f-749b-4491-8b16-7c072e2714f9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7d799877b9-bjwnb\" (UID: \"db87c04f-749b-4491-8b16-7c072e2714f9\") " pod="opendatahub/opendatahub-operator-controller-manager-7d799877b9-bjwnb" May 27 21:36:54.968486 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:54.968455 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/db87c04f-749b-4491-8b16-7c072e2714f9-webhook-cert\") pod \"opendatahub-operator-controller-manager-7d799877b9-bjwnb\" (UID: \"db87c04f-749b-4491-8b16-7c072e2714f9\") " pod="opendatahub/opendatahub-operator-controller-manager-7d799877b9-bjwnb" May 27 21:36:54.968608 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:54.968495 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8q6pl\" (UniqueName: \"kubernetes.io/projected/db87c04f-749b-4491-8b16-7c072e2714f9-kube-api-access-8q6pl\") pod \"opendatahub-operator-controller-manager-7d799877b9-bjwnb\" (UID: \"db87c04f-749b-4491-8b16-7c072e2714f9\") " pod="opendatahub/opendatahub-operator-controller-manager-7d799877b9-bjwnb" May 27 21:36:54.968608 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:54.968548 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/db87c04f-749b-4491-8b16-7c072e2714f9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7d799877b9-bjwnb\" (UID: \"db87c04f-749b-4491-8b16-7c072e2714f9\") " pod="opendatahub/opendatahub-operator-controller-manager-7d799877b9-bjwnb" May 27 21:36:54.970843 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:54.970819 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/db87c04f-749b-4491-8b16-7c072e2714f9-webhook-cert\") pod \"opendatahub-operator-controller-manager-7d799877b9-bjwnb\" (UID: \"db87c04f-749b-4491-8b16-7c072e2714f9\") " pod="opendatahub/opendatahub-operator-controller-manager-7d799877b9-bjwnb" May 27 21:36:54.970939 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:54.970850 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/db87c04f-749b-4491-8b16-7c072e2714f9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7d799877b9-bjwnb\" (UID: \"db87c04f-749b-4491-8b16-7c072e2714f9\") " pod="opendatahub/opendatahub-operator-controller-manager-7d799877b9-bjwnb" May 27 21:36:54.981867 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:54.981845 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8q6pl\" (UniqueName: \"kubernetes.io/projected/db87c04f-749b-4491-8b16-7c072e2714f9-kube-api-access-8q6pl\") pod \"opendatahub-operator-controller-manager-7d799877b9-bjwnb\" (UID: \"db87c04f-749b-4491-8b16-7c072e2714f9\") " pod="opendatahub/opendatahub-operator-controller-manager-7d799877b9-bjwnb" May 27 21:36:55.059083 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:55.058992 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7d799877b9-bjwnb" May 27 21:36:55.203016 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:55.202978 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7d799877b9-bjwnb"] May 27 21:36:55.205365 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:36:55.205337 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb87c04f_749b_4491_8b16_7c072e2714f9.slice/crio-dbde701ffff051588c1060dcefcf430d2d9fd032f314481e55c2075dd54d3d23 WatchSource:0}: Error finding container dbde701ffff051588c1060dcefcf430d2d9fd032f314481e55c2075dd54d3d23: Status 404 returned error can't find the container with id dbde701ffff051588c1060dcefcf430d2d9fd032f314481e55c2075dd54d3d23 May 27 21:36:55.765005 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:55.764969 2564 generic.go:358] "Generic (PLEG): container finished" podID="88de7aeb-b93b-4f17-91f9-a0489a81edde" containerID="a307e60dd2b97fe9c381fc1a3e998bef230559286b9e3343b5c7de9a2bfd3b73" exitCode=0 May 27 21:36:55.765490 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:55.765040 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c9bs8t5" event={"ID":"88de7aeb-b93b-4f17-91f9-a0489a81edde","Type":"ContainerDied","Data":"a307e60dd2b97fe9c381fc1a3e998bef230559286b9e3343b5c7de9a2bfd3b73"} May 27 21:36:55.766249 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:55.766226 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7d799877b9-bjwnb" event={"ID":"db87c04f-749b-4491-8b16-7c072e2714f9","Type":"ContainerStarted","Data":"dbde701ffff051588c1060dcefcf430d2d9fd032f314481e55c2075dd54d3d23"} May 27 21:36:57.818784 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:57.818762 2564 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c9bs8t5" May 27 21:36:57.894109 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:57.894088 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/88de7aeb-b93b-4f17-91f9-a0489a81edde-util\") pod \"88de7aeb-b93b-4f17-91f9-a0489a81edde\" (UID: \"88de7aeb-b93b-4f17-91f9-a0489a81edde\") " May 27 21:36:57.894219 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:57.894195 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/88de7aeb-b93b-4f17-91f9-a0489a81edde-bundle\") pod \"88de7aeb-b93b-4f17-91f9-a0489a81edde\" (UID: \"88de7aeb-b93b-4f17-91f9-a0489a81edde\") " May 27 21:36:57.894290 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:57.894254 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h7pkv\" (UniqueName: \"kubernetes.io/projected/88de7aeb-b93b-4f17-91f9-a0489a81edde-kube-api-access-h7pkv\") pod \"88de7aeb-b93b-4f17-91f9-a0489a81edde\" (UID: \"88de7aeb-b93b-4f17-91f9-a0489a81edde\") " May 27 21:36:57.894975 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:57.894932 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88de7aeb-b93b-4f17-91f9-a0489a81edde-bundle" (OuterVolumeSpecName: "bundle") pod "88de7aeb-b93b-4f17-91f9-a0489a81edde" (UID: "88de7aeb-b93b-4f17-91f9-a0489a81edde"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 27 21:36:57.896532 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:57.896381 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88de7aeb-b93b-4f17-91f9-a0489a81edde-kube-api-access-h7pkv" (OuterVolumeSpecName: "kube-api-access-h7pkv") pod "88de7aeb-b93b-4f17-91f9-a0489a81edde" (UID: "88de7aeb-b93b-4f17-91f9-a0489a81edde"). InnerVolumeSpecName "kube-api-access-h7pkv". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 27 21:36:57.901990 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:57.901949 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88de7aeb-b93b-4f17-91f9-a0489a81edde-util" (OuterVolumeSpecName: "util") pod "88de7aeb-b93b-4f17-91f9-a0489a81edde" (UID: "88de7aeb-b93b-4f17-91f9-a0489a81edde"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 27 21:36:57.995931 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:57.995894 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/88de7aeb-b93b-4f17-91f9-a0489a81edde-bundle\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:36:57.995931 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:57.995926 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h7pkv\" (UniqueName: \"kubernetes.io/projected/88de7aeb-b93b-4f17-91f9-a0489a81edde-kube-api-access-h7pkv\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:36:57.995931 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:57.995938 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/88de7aeb-b93b-4f17-91f9-a0489a81edde-util\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:36:58.779065 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:58.779027 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c9bs8t5" event={"ID":"88de7aeb-b93b-4f17-91f9-a0489a81edde","Type":"ContainerDied","Data":"fadef1bbe2ab41a1134443a09bea1a8f7e3e6bbbf9ae4b6af6033c46ef7fc1c4"} May 27 21:36:58.779065 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:58.779062 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fadef1bbe2ab41a1134443a09bea1a8f7e3e6bbbf9ae4b6af6033c46ef7fc1c4" May 27 21:36:58.779307 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:58.779078 2564 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c9bs8t5" May 27 21:36:58.782076 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:58.781399 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7d799877b9-bjwnb" event={"ID":"db87c04f-749b-4491-8b16-7c072e2714f9","Type":"ContainerStarted","Data":"816bb315f37a309aa6c79a87957264dbd261bf9427c9f43cc61711acaf9b728b"} May 27 21:36:58.782198 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:58.782135 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7d799877b9-bjwnb" May 27 21:36:58.812021 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:36:58.811977 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7d799877b9-bjwnb" podStartSLOduration=2.169427183 podStartE2EDuration="4.811964827s" podCreationTimestamp="2026-05-27 21:36:54 +0000 UTC" firstStartedPulling="2026-05-27 21:36:55.207164108 +0000 UTC m=+488.681368817" lastFinishedPulling="2026-05-27 21:36:57.84970175 +0000 UTC m=+491.323906461" observedRunningTime="2026-05-27 21:36:58.809661027 +0000 UTC m=+492.283865756" watchObservedRunningTime="2026-05-27 21:36:58.811964827 +0000 UTC m=+492.286169557" May 27 21:37:10.791012 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:10.790982 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7d799877b9-bjwnb" May 27 21:37:13.117204 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:13.117164 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835qtc8w"] May 27 21:37:13.117627 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:13.117611 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88de7aeb-b93b-4f17-91f9-a0489a81edde" containerName="util" May 27 21:37:13.117665 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:13.117630 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="88de7aeb-b93b-4f17-91f9-a0489a81edde" containerName="util" May 27 21:37:13.117665 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:13.117646 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88de7aeb-b93b-4f17-91f9-a0489a81edde" containerName="extract" May 27 21:37:13.117665 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:13.117655 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="88de7aeb-b93b-4f17-91f9-a0489a81edde" containerName="extract" May 27 21:37:13.117809 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:13.117687 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88de7aeb-b93b-4f17-91f9-a0489a81edde" containerName="pull" May 27 21:37:13.117809 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:13.117696 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="88de7aeb-b93b-4f17-91f9-a0489a81edde" containerName="pull" May 27 21:37:13.117809 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:13.117770 2564 memory_manager.go:356] "RemoveStaleState removing state" podUID="88de7aeb-b93b-4f17-91f9-a0489a81edde" containerName="extract" May 27 21:37:13.121260 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:13.121235 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835qtc8w" May 27 21:37:13.124460 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:13.124440 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" May 27 21:37:13.124616 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:13.124598 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-tmqch\"" May 27 21:37:13.125618 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:13.125601 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" May 27 21:37:13.129937 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:13.129903 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835qtc8w"] May 27 21:37:13.222448 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:13.222402 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/58295c74-4459-4eed-86d6-fcb17b7094c2-bundle\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835qtc8w\" (UID: \"58295c74-4459-4eed-86d6-fcb17b7094c2\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835qtc8w" May 27 21:37:13.222632 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:13.222464 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/58295c74-4459-4eed-86d6-fcb17b7094c2-util\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835qtc8w\" (UID: \"58295c74-4459-4eed-86d6-fcb17b7094c2\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835qtc8w" May 27 21:37:13.222632 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:13.222541 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4krr\" (UniqueName: \"kubernetes.io/projected/58295c74-4459-4eed-86d6-fcb17b7094c2-kube-api-access-p4krr\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835qtc8w\" (UID: \"58295c74-4459-4eed-86d6-fcb17b7094c2\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835qtc8w" May 27 21:37:13.323839 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:13.323804 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p4krr\" (UniqueName: \"kubernetes.io/projected/58295c74-4459-4eed-86d6-fcb17b7094c2-kube-api-access-p4krr\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835qtc8w\" (UID: \"58295c74-4459-4eed-86d6-fcb17b7094c2\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835qtc8w" May 27 21:37:13.324053 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:13.323868 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/58295c74-4459-4eed-86d6-fcb17b7094c2-bundle\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835qtc8w\" (UID: \"58295c74-4459-4eed-86d6-fcb17b7094c2\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835qtc8w" May 27 21:37:13.324053 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:13.323903 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/58295c74-4459-4eed-86d6-fcb17b7094c2-util\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835qtc8w\" (UID: \"58295c74-4459-4eed-86d6-fcb17b7094c2\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835qtc8w" May 27 21:37:13.324323 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:13.324295 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/58295c74-4459-4eed-86d6-fcb17b7094c2-bundle\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835qtc8w\" (UID: \"58295c74-4459-4eed-86d6-fcb17b7094c2\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835qtc8w" May 27 21:37:13.324396 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:13.324321 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/58295c74-4459-4eed-86d6-fcb17b7094c2-util\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835qtc8w\" (UID: \"58295c74-4459-4eed-86d6-fcb17b7094c2\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835qtc8w" May 27 21:37:13.332602 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:13.332575 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4krr\" (UniqueName: \"kubernetes.io/projected/58295c74-4459-4eed-86d6-fcb17b7094c2-kube-api-access-p4krr\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835qtc8w\" (UID: \"58295c74-4459-4eed-86d6-fcb17b7094c2\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835qtc8w" May 27 21:37:13.432174 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:13.432136 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835qtc8w" May 27 21:37:13.589203 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:13.589176 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835qtc8w"] May 27 21:37:13.591973 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:37:13.591944 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58295c74_4459_4eed_86d6_fcb17b7094c2.slice/crio-a5dfddc89cfbc16d49fd8b04337bc1b85e043619afad3aff52bd5f13f8144212 WatchSource:0}: Error finding container a5dfddc89cfbc16d49fd8b04337bc1b85e043619afad3aff52bd5f13f8144212: Status 404 returned error can't find the container with id a5dfddc89cfbc16d49fd8b04337bc1b85e043619afad3aff52bd5f13f8144212 May 27 21:37:13.833291 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:13.833242 2564 generic.go:358] "Generic (PLEG): container finished" podID="58295c74-4459-4eed-86d6-fcb17b7094c2" containerID="3e73a038ae0b81199d28ba4bc0e5ef96d1db7b2212146082abcd28d9d3b59da5" exitCode=0 May 27 21:37:13.833489 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:13.833314 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835qtc8w" event={"ID":"58295c74-4459-4eed-86d6-fcb17b7094c2","Type":"ContainerDied","Data":"3e73a038ae0b81199d28ba4bc0e5ef96d1db7b2212146082abcd28d9d3b59da5"} May 27 21:37:13.833489 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:13.833342 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835qtc8w" event={"ID":"58295c74-4459-4eed-86d6-fcb17b7094c2","Type":"ContainerStarted","Data":"a5dfddc89cfbc16d49fd8b04337bc1b85e043619afad3aff52bd5f13f8144212"} May 27 21:37:14.325668 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:14.325626 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-68b96d7ccc-ghzfh"] May 27 21:37:14.329119 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:14.329095 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-68b96d7ccc-ghzfh" May 27 21:37:14.332891 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:14.332865 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-j6pnc\"" May 27 21:37:14.333026 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:14.332900 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" May 27 21:37:14.333026 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:14.332907 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" May 27 21:37:14.345728 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:14.345702 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-68b96d7ccc-ghzfh"] May 27 21:37:14.432133 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:14.432110 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f1965d5f-a8e4-4a0b-9e0b-4596c90ab37f-tmp\") pod \"kube-auth-proxy-68b96d7ccc-ghzfh\" (UID: \"f1965d5f-a8e4-4a0b-9e0b-4596c90ab37f\") " pod="openshift-ingress/kube-auth-proxy-68b96d7ccc-ghzfh" May 27 21:37:14.432219 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:14.432144 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1965d5f-a8e4-4a0b-9e0b-4596c90ab37f-tls-certs\") pod \"kube-auth-proxy-68b96d7ccc-ghzfh\" (UID: \"f1965d5f-a8e4-4a0b-9e0b-4596c90ab37f\") " pod="openshift-ingress/kube-auth-proxy-68b96d7ccc-ghzfh" May 27 21:37:14.432219 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:14.432162 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqdxm\" (UniqueName: \"kubernetes.io/projected/f1965d5f-a8e4-4a0b-9e0b-4596c90ab37f-kube-api-access-mqdxm\") pod \"kube-auth-proxy-68b96d7ccc-ghzfh\" (UID: \"f1965d5f-a8e4-4a0b-9e0b-4596c90ab37f\") " pod="openshift-ingress/kube-auth-proxy-68b96d7ccc-ghzfh" May 27 21:37:14.533354 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:14.533328 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f1965d5f-a8e4-4a0b-9e0b-4596c90ab37f-tmp\") pod \"kube-auth-proxy-68b96d7ccc-ghzfh\" (UID: \"f1965d5f-a8e4-4a0b-9e0b-4596c90ab37f\") " pod="openshift-ingress/kube-auth-proxy-68b96d7ccc-ghzfh" May 27 21:37:14.533453 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:14.533369 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1965d5f-a8e4-4a0b-9e0b-4596c90ab37f-tls-certs\") pod \"kube-auth-proxy-68b96d7ccc-ghzfh\" (UID: \"f1965d5f-a8e4-4a0b-9e0b-4596c90ab37f\") " pod="openshift-ingress/kube-auth-proxy-68b96d7ccc-ghzfh" May 27 21:37:14.533499 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:14.533484 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mqdxm\" (UniqueName: \"kubernetes.io/projected/f1965d5f-a8e4-4a0b-9e0b-4596c90ab37f-kube-api-access-mqdxm\") pod \"kube-auth-proxy-68b96d7ccc-ghzfh\" (UID: \"f1965d5f-a8e4-4a0b-9e0b-4596c90ab37f\") " pod="openshift-ingress/kube-auth-proxy-68b96d7ccc-ghzfh" May 27 21:37:14.535477 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:14.535459 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f1965d5f-a8e4-4a0b-9e0b-4596c90ab37f-tmp\") pod \"kube-auth-proxy-68b96d7ccc-ghzfh\" (UID: \"f1965d5f-a8e4-4a0b-9e0b-4596c90ab37f\") " pod="openshift-ingress/kube-auth-proxy-68b96d7ccc-ghzfh" May 27 21:37:14.535588 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:14.535573 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1965d5f-a8e4-4a0b-9e0b-4596c90ab37f-tls-certs\") pod \"kube-auth-proxy-68b96d7ccc-ghzfh\" (UID: \"f1965d5f-a8e4-4a0b-9e0b-4596c90ab37f\") " pod="openshift-ingress/kube-auth-proxy-68b96d7ccc-ghzfh" May 27 21:37:14.542009 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:14.541981 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqdxm\" (UniqueName: \"kubernetes.io/projected/f1965d5f-a8e4-4a0b-9e0b-4596c90ab37f-kube-api-access-mqdxm\") pod \"kube-auth-proxy-68b96d7ccc-ghzfh\" (UID: \"f1965d5f-a8e4-4a0b-9e0b-4596c90ab37f\") " pod="openshift-ingress/kube-auth-proxy-68b96d7ccc-ghzfh" May 27 21:37:14.649459 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:14.649375 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-68b96d7ccc-ghzfh" May 27 21:37:14.784186 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:14.784049 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-68b96d7ccc-ghzfh"] May 27 21:37:14.786704 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:37:14.786679 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1965d5f_a8e4_4a0b_9e0b_4596c90ab37f.slice/crio-3a341ee1236c470799d7eafa886e3c124687d4616863dbaa5a5292c90adf7ab5 WatchSource:0}: Error finding container 3a341ee1236c470799d7eafa886e3c124687d4616863dbaa5a5292c90adf7ab5: Status 404 returned error can't find the container with id 3a341ee1236c470799d7eafa886e3c124687d4616863dbaa5a5292c90adf7ab5 May 27 21:37:14.837506 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:14.837457 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-68b96d7ccc-ghzfh" event={"ID":"f1965d5f-a8e4-4a0b-9e0b-4596c90ab37f","Type":"ContainerStarted","Data":"3a341ee1236c470799d7eafa886e3c124687d4616863dbaa5a5292c90adf7ab5"} May 27 21:37:14.839286 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:14.839234 2564 generic.go:358] "Generic (PLEG): container finished" podID="58295c74-4459-4eed-86d6-fcb17b7094c2" containerID="981f3f0129ea0d4c294dd1cf1b6b3cec1706414a6b127308a62aa23b3c89d7ff" exitCode=0 May 27 21:37:14.839286 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:14.839282 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835qtc8w" event={"ID":"58295c74-4459-4eed-86d6-fcb17b7094c2","Type":"ContainerDied","Data":"981f3f0129ea0d4c294dd1cf1b6b3cec1706414a6b127308a62aa23b3c89d7ff"} May 27 21:37:15.846364 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:15.846315 2564 generic.go:358] "Generic (PLEG): container finished" podID="58295c74-4459-4eed-86d6-fcb17b7094c2" containerID="ead370c2b45e504505197ebcb292b43a747206be5565dd9171f2a55f6b2a5702" exitCode=0 May 27 21:37:15.846888 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:15.846477 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835qtc8w" event={"ID":"58295c74-4459-4eed-86d6-fcb17b7094c2","Type":"ContainerDied","Data":"ead370c2b45e504505197ebcb292b43a747206be5565dd9171f2a55f6b2a5702"} May 27 21:37:16.576499 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:16.576455 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-858dbf95b8-tn94s"] May 27 21:37:16.589283 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:16.589234 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-858dbf95b8-tn94s"] May 27 21:37:16.589464 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:16.589415 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-858dbf95b8-tn94s" May 27 21:37:16.592058 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:16.591991 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" May 27 21:37:16.592058 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:16.592033 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-sglct\"" May 27 21:37:16.752907 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:16.752864 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krxs2\" (UniqueName: \"kubernetes.io/projected/6333b9d0-8455-4556-adde-669e2259a722-kube-api-access-krxs2\") pod \"odh-model-controller-858dbf95b8-tn94s\" (UID: \"6333b9d0-8455-4556-adde-669e2259a722\") " pod="opendatahub/odh-model-controller-858dbf95b8-tn94s" May 27 21:37:16.753073 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:16.752922 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6333b9d0-8455-4556-adde-669e2259a722-cert\") pod \"odh-model-controller-858dbf95b8-tn94s\" (UID: \"6333b9d0-8455-4556-adde-669e2259a722\") " pod="opendatahub/odh-model-controller-858dbf95b8-tn94s" May 27 21:37:16.853469 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:16.853432 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-krxs2\" (UniqueName: \"kubernetes.io/projected/6333b9d0-8455-4556-adde-669e2259a722-kube-api-access-krxs2\") pod \"odh-model-controller-858dbf95b8-tn94s\" (UID: \"6333b9d0-8455-4556-adde-669e2259a722\") " pod="opendatahub/odh-model-controller-858dbf95b8-tn94s" May 27 21:37:16.853911 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:16.853485 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6333b9d0-8455-4556-adde-669e2259a722-cert\") pod \"odh-model-controller-858dbf95b8-tn94s\" (UID: \"6333b9d0-8455-4556-adde-669e2259a722\") " pod="opendatahub/odh-model-controller-858dbf95b8-tn94s" May 27 21:37:16.853911 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:37:16.853688 2564 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found May 27 21:37:16.853911 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:37:16.853761 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6333b9d0-8455-4556-adde-669e2259a722-cert podName:6333b9d0-8455-4556-adde-669e2259a722 nodeName:}" failed. No retries permitted until 2026-05-27 21:37:17.353739419 +0000 UTC m=+510.827944134 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6333b9d0-8455-4556-adde-669e2259a722-cert") pod "odh-model-controller-858dbf95b8-tn94s" (UID: "6333b9d0-8455-4556-adde-669e2259a722") : secret "odh-model-controller-webhook-cert" not found May 27 21:37:16.872577 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:16.872503 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-krxs2\" (UniqueName: \"kubernetes.io/projected/6333b9d0-8455-4556-adde-669e2259a722-kube-api-access-krxs2\") pod \"odh-model-controller-858dbf95b8-tn94s\" (UID: \"6333b9d0-8455-4556-adde-669e2259a722\") " pod="opendatahub/odh-model-controller-858dbf95b8-tn94s" May 27 21:37:17.010653 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:17.010622 2564 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835qtc8w" May 27 21:37:17.157382 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:17.157277 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/58295c74-4459-4eed-86d6-fcb17b7094c2-util\") pod \"58295c74-4459-4eed-86d6-fcb17b7094c2\" (UID: \"58295c74-4459-4eed-86d6-fcb17b7094c2\") " May 27 21:37:17.157382 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:17.157359 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p4krr\" (UniqueName: \"kubernetes.io/projected/58295c74-4459-4eed-86d6-fcb17b7094c2-kube-api-access-p4krr\") pod \"58295c74-4459-4eed-86d6-fcb17b7094c2\" (UID: \"58295c74-4459-4eed-86d6-fcb17b7094c2\") " May 27 21:37:17.157614 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:17.157395 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/58295c74-4459-4eed-86d6-fcb17b7094c2-bundle\") pod \"58295c74-4459-4eed-86d6-fcb17b7094c2\" (UID: \"58295c74-4459-4eed-86d6-fcb17b7094c2\") " May 27 21:37:17.158511 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:17.158480 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58295c74-4459-4eed-86d6-fcb17b7094c2-bundle" (OuterVolumeSpecName: "bundle") pod "58295c74-4459-4eed-86d6-fcb17b7094c2" (UID: "58295c74-4459-4eed-86d6-fcb17b7094c2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 27 21:37:17.159646 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:17.159616 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58295c74-4459-4eed-86d6-fcb17b7094c2-kube-api-access-p4krr" (OuterVolumeSpecName: "kube-api-access-p4krr") pod "58295c74-4459-4eed-86d6-fcb17b7094c2" (UID: "58295c74-4459-4eed-86d6-fcb17b7094c2"). InnerVolumeSpecName "kube-api-access-p4krr". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 27 21:37:17.163729 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:17.163603 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58295c74-4459-4eed-86d6-fcb17b7094c2-util" (OuterVolumeSpecName: "util") pod "58295c74-4459-4eed-86d6-fcb17b7094c2" (UID: "58295c74-4459-4eed-86d6-fcb17b7094c2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 27 21:37:17.258900 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:17.258861 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/58295c74-4459-4eed-86d6-fcb17b7094c2-util\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:37:17.258900 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:17.258900 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p4krr\" (UniqueName: \"kubernetes.io/projected/58295c74-4459-4eed-86d6-fcb17b7094c2-kube-api-access-p4krr\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:37:17.259103 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:17.258914 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/58295c74-4459-4eed-86d6-fcb17b7094c2-bundle\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:37:17.359715 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:17.359673 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6333b9d0-8455-4556-adde-669e2259a722-cert\") pod \"odh-model-controller-858dbf95b8-tn94s\" (UID: \"6333b9d0-8455-4556-adde-669e2259a722\") " pod="opendatahub/odh-model-controller-858dbf95b8-tn94s" May 27 21:37:17.362560 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:17.362531 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6333b9d0-8455-4556-adde-669e2259a722-cert\") pod \"odh-model-controller-858dbf95b8-tn94s\" (UID: \"6333b9d0-8455-4556-adde-669e2259a722\") " pod="opendatahub/odh-model-controller-858dbf95b8-tn94s" May 27 21:37:17.504902 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:17.504862 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-858dbf95b8-tn94s" May 27 21:37:17.856983 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:17.856892 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835qtc8w" event={"ID":"58295c74-4459-4eed-86d6-fcb17b7094c2","Type":"ContainerDied","Data":"a5dfddc89cfbc16d49fd8b04337bc1b85e043619afad3aff52bd5f13f8144212"} May 27 21:37:17.856983 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:17.856938 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a5dfddc89cfbc16d49fd8b04337bc1b85e043619afad3aff52bd5f13f8144212" May 27 21:37:17.857496 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:17.857145 2564 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835qtc8w" May 27 21:37:18.112777 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:18.112749 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-858dbf95b8-tn94s"] May 27 21:37:18.115071 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:37:18.115037 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6333b9d0_8455_4556_adde_669e2259a722.slice/crio-5b7a0eca4089ab10685824e2805a813e99949b15d665b23806396874c84af2f2 WatchSource:0}: Error finding container 5b7a0eca4089ab10685824e2805a813e99949b15d665b23806396874c84af2f2: Status 404 returned error can't find the container with id 5b7a0eca4089ab10685824e2805a813e99949b15d665b23806396874c84af2f2 May 27 21:37:18.863927 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:18.863886 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-68b96d7ccc-ghzfh" event={"ID":"f1965d5f-a8e4-4a0b-9e0b-4596c90ab37f","Type":"ContainerStarted","Data":"dae2f072330d64975d3f4f5f21b8aa2ad2183f7822137d50b27d9143f0010564"} May 27 21:37:18.866453 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:18.866422 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-tn94s" event={"ID":"6333b9d0-8455-4556-adde-669e2259a722","Type":"ContainerStarted","Data":"5b7a0eca4089ab10685824e2805a813e99949b15d665b23806396874c84af2f2"} May 27 21:37:18.883083 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:18.883023 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-68b96d7ccc-ghzfh" podStartSLOduration=1.641851764 podStartE2EDuration="4.883002512s" podCreationTimestamp="2026-05-27 21:37:14 +0000 UTC" firstStartedPulling="2026-05-27 21:37:14.788504679 +0000 UTC m=+508.262709387" lastFinishedPulling="2026-05-27 21:37:18.029655425 +0000 UTC m=+511.503860135" observedRunningTime="2026-05-27 21:37:18.881426977 +0000 UTC m=+512.355631708" watchObservedRunningTime="2026-05-27 21:37:18.883002512 +0000 UTC m=+512.357207247" May 27 21:37:20.878172 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:20.878076 2564 generic.go:358] "Generic (PLEG): container finished" podID="6333b9d0-8455-4556-adde-669e2259a722" containerID="1fb9f6673271cbd33f3201cedc261e90c8df062f0e53afbfaae5d4d5d29ba054" exitCode=1 May 27 21:37:20.878567 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:20.878168 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-tn94s" event={"ID":"6333b9d0-8455-4556-adde-669e2259a722","Type":"ContainerDied","Data":"1fb9f6673271cbd33f3201cedc261e90c8df062f0e53afbfaae5d4d5d29ba054"} May 27 21:37:20.878567 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:20.878439 2564 scope.go:117] "RemoveContainer" containerID="1fb9f6673271cbd33f3201cedc261e90c8df062f0e53afbfaae5d4d5d29ba054" May 27 21:37:21.883884 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:21.883848 2564 generic.go:358] "Generic (PLEG): container finished" podID="6333b9d0-8455-4556-adde-669e2259a722" containerID="18ac0ee028321c751f9e80d72eeab36bb660ea77afd284c563bb8437be5c592d" exitCode=1 May 27 21:37:21.884383 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:21.883894 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-tn94s" event={"ID":"6333b9d0-8455-4556-adde-669e2259a722","Type":"ContainerDied","Data":"18ac0ee028321c751f9e80d72eeab36bb660ea77afd284c563bb8437be5c592d"} May 27 21:37:21.884383 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:21.883925 2564 scope.go:117] "RemoveContainer" containerID="1fb9f6673271cbd33f3201cedc261e90c8df062f0e53afbfaae5d4d5d29ba054" May 27 21:37:21.884383 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:21.884230 2564 scope.go:117] "RemoveContainer" containerID="18ac0ee028321c751f9e80d72eeab36bb660ea77afd284c563bb8437be5c592d" May 27 21:37:21.884535 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:37:21.884497 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=odh-model-controller-858dbf95b8-tn94s_opendatahub(6333b9d0-8455-4556-adde-669e2259a722)\"" pod="opendatahub/odh-model-controller-858dbf95b8-tn94s" podUID="6333b9d0-8455-4556-adde-669e2259a722" May 27 21:37:22.539376 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:22.539340 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a8c5h8"] May 27 21:37:22.539690 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:22.539677 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58295c74-4459-4eed-86d6-fcb17b7094c2" containerName="extract" May 27 21:37:22.539743 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:22.539691 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="58295c74-4459-4eed-86d6-fcb17b7094c2" containerName="extract" May 27 21:37:22.539743 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:22.539710 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58295c74-4459-4eed-86d6-fcb17b7094c2" containerName="util" May 27 21:37:22.539743 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:22.539715 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="58295c74-4459-4eed-86d6-fcb17b7094c2" containerName="util" May 27 21:37:22.539743 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:22.539722 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58295c74-4459-4eed-86d6-fcb17b7094c2" containerName="pull" May 27 21:37:22.539743 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:22.539728 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="58295c74-4459-4eed-86d6-fcb17b7094c2" containerName="pull" May 27 21:37:22.539896 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:22.539785 2564 memory_manager.go:356] "RemoveStaleState removing state" podUID="58295c74-4459-4eed-86d6-fcb17b7094c2" containerName="extract" May 27 21:37:22.543886 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:22.543864 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a8c5h8" May 27 21:37:22.551650 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:22.551627 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" May 27 21:37:22.553027 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:22.553007 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" May 27 21:37:22.553282 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:22.553254 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-tmqch\"" May 27 21:37:22.580951 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:22.580915 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a8c5h8"] May 27 21:37:22.711062 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:22.710766 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmqjs\" (UniqueName: \"kubernetes.io/projected/64699393-17ae-460f-ab95-a1f75fc2d7e7-kube-api-access-wmqjs\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a8c5h8\" (UID: \"64699393-17ae-460f-ab95-a1f75fc2d7e7\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a8c5h8" May 27 21:37:22.711062 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:22.710879 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/64699393-17ae-460f-ab95-a1f75fc2d7e7-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a8c5h8\" (UID: \"64699393-17ae-460f-ab95-a1f75fc2d7e7\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a8c5h8" May 27 21:37:22.711062 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:22.710950 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/64699393-17ae-460f-ab95-a1f75fc2d7e7-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a8c5h8\" (UID: \"64699393-17ae-460f-ab95-a1f75fc2d7e7\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a8c5h8" May 27 21:37:22.811778 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:22.811681 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/64699393-17ae-460f-ab95-a1f75fc2d7e7-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a8c5h8\" (UID: \"64699393-17ae-460f-ab95-a1f75fc2d7e7\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a8c5h8" May 27 21:37:22.812062 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:22.812037 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/64699393-17ae-460f-ab95-a1f75fc2d7e7-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a8c5h8\" (UID: \"64699393-17ae-460f-ab95-a1f75fc2d7e7\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a8c5h8" May 27 21:37:22.812575 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:22.812552 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wmqjs\" (UniqueName: \"kubernetes.io/projected/64699393-17ae-460f-ab95-a1f75fc2d7e7-kube-api-access-wmqjs\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a8c5h8\" (UID: \"64699393-17ae-460f-ab95-a1f75fc2d7e7\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a8c5h8" May 27 21:37:22.813054 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:22.812463 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/64699393-17ae-460f-ab95-a1f75fc2d7e7-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a8c5h8\" (UID: \"64699393-17ae-460f-ab95-a1f75fc2d7e7\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a8c5h8" May 27 21:37:22.813172 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:22.812331 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/64699393-17ae-460f-ab95-a1f75fc2d7e7-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a8c5h8\" (UID: \"64699393-17ae-460f-ab95-a1f75fc2d7e7\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a8c5h8" May 27 21:37:22.832410 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:22.832388 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmqjs\" (UniqueName: \"kubernetes.io/projected/64699393-17ae-460f-ab95-a1f75fc2d7e7-kube-api-access-wmqjs\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a8c5h8\" (UID: \"64699393-17ae-460f-ab95-a1f75fc2d7e7\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a8c5h8" May 27 21:37:22.853937 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:22.853907 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a8c5h8" May 27 21:37:22.891875 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:22.891845 2564 scope.go:117] "RemoveContainer" containerID="18ac0ee028321c751f9e80d72eeab36bb660ea77afd284c563bb8437be5c592d" May 27 21:37:22.892301 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:37:22.892124 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=odh-model-controller-858dbf95b8-tn94s_opendatahub(6333b9d0-8455-4556-adde-669e2259a722)\"" pod="opendatahub/odh-model-controller-858dbf95b8-tn94s" podUID="6333b9d0-8455-4556-adde-669e2259a722" May 27 21:37:23.037496 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:23.037464 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a8c5h8"] May 27 21:37:23.038790 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:37:23.038760 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64699393_17ae_460f_ab95_a1f75fc2d7e7.slice/crio-bf7c1827279748f7b5013c7e07758e6dab5440ce7d41c8c78a1506ec765ff2c0 WatchSource:0}: Error finding container bf7c1827279748f7b5013c7e07758e6dab5440ce7d41c8c78a1506ec765ff2c0: Status 404 returned error can't find the container with id bf7c1827279748f7b5013c7e07758e6dab5440ce7d41c8c78a1506ec765ff2c0 May 27 21:37:23.896532 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:23.896498 2564 generic.go:358] "Generic (PLEG): container finished" podID="64699393-17ae-460f-ab95-a1f75fc2d7e7" containerID="fd0ed0ed1fd70a046b435d1dc08b903028242b1eb6814489e0022498140957b6" exitCode=0 May 27 21:37:23.896950 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:23.896593 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a8c5h8" event={"ID":"64699393-17ae-460f-ab95-a1f75fc2d7e7","Type":"ContainerDied","Data":"fd0ed0ed1fd70a046b435d1dc08b903028242b1eb6814489e0022498140957b6"} May 27 21:37:23.896950 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:23.896638 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a8c5h8" event={"ID":"64699393-17ae-460f-ab95-a1f75fc2d7e7","Type":"ContainerStarted","Data":"bf7c1827279748f7b5013c7e07758e6dab5440ce7d41c8c78a1506ec765ff2c0"} May 27 21:37:25.658882 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:25.658845 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-856948b99f-vjsc4"] May 27 21:37:25.662958 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:25.662937 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-856948b99f-vjsc4" May 27 21:37:25.677949 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:25.677927 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-v7svl\"" May 27 21:37:25.688105 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:25.688072 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" May 27 21:37:25.692349 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:25.692316 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-856948b99f-vjsc4"] May 27 21:37:25.817762 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:25.817730 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-vzw72"] May 27 21:37:25.821201 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:25.821181 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-55f49c5f94-vzw72" May 27 21:37:25.832426 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:25.832405 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"kube-root-ca.crt\"" May 27 21:37:25.832426 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:25.832421 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"openshift-service-ca.crt\"" May 27 21:37:25.835669 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:25.835642 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-operators\"/\"servicemesh-operator3-dockercfg-ggvwq\"" May 27 21:37:25.838550 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:25.838531 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wft4k\" (UniqueName: \"kubernetes.io/projected/7ef19ce3-6eb8-4064-9c5e-31eb60b8c56a-kube-api-access-wft4k\") pod \"kserve-controller-manager-856948b99f-vjsc4\" (UID: \"7ef19ce3-6eb8-4064-9c5e-31eb60b8c56a\") " pod="opendatahub/kserve-controller-manager-856948b99f-vjsc4" May 27 21:37:25.838633 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:25.838570 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7ef19ce3-6eb8-4064-9c5e-31eb60b8c56a-cert\") pod \"kserve-controller-manager-856948b99f-vjsc4\" (UID: \"7ef19ce3-6eb8-4064-9c5e-31eb60b8c56a\") " pod="opendatahub/kserve-controller-manager-856948b99f-vjsc4" May 27 21:37:25.910063 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:25.909973 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-vzw72"] May 27 21:37:25.910372 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:25.910344 2564 generic.go:358] "Generic (PLEG): container finished" podID="64699393-17ae-460f-ab95-a1f75fc2d7e7" containerID="09be74311fc122fcbcb237545854b80ab1ccb8d79372889672c79f314f28eefb" exitCode=0 May 27 21:37:25.910466 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:25.910377 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a8c5h8" event={"ID":"64699393-17ae-460f-ab95-a1f75fc2d7e7","Type":"ContainerDied","Data":"09be74311fc122fcbcb237545854b80ab1ccb8d79372889672c79f314f28eefb"} May 27 21:37:25.944311 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:25.941201 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7ef19ce3-6eb8-4064-9c5e-31eb60b8c56a-cert\") pod \"kserve-controller-manager-856948b99f-vjsc4\" (UID: \"7ef19ce3-6eb8-4064-9c5e-31eb60b8c56a\") " pod="opendatahub/kserve-controller-manager-856948b99f-vjsc4" May 27 21:37:25.944311 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:25.941309 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4lnw\" (UniqueName: \"kubernetes.io/projected/5b9e9a6e-a950-496c-937a-f6563ee83fc5-kube-api-access-c4lnw\") pod \"servicemesh-operator3-55f49c5f94-vzw72\" (UID: \"5b9e9a6e-a950-496c-937a-f6563ee83fc5\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-vzw72" May 27 21:37:25.944311 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:25.941377 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wft4k\" (UniqueName: \"kubernetes.io/projected/7ef19ce3-6eb8-4064-9c5e-31eb60b8c56a-kube-api-access-wft4k\") pod \"kserve-controller-manager-856948b99f-vjsc4\" (UID: \"7ef19ce3-6eb8-4064-9c5e-31eb60b8c56a\") " pod="opendatahub/kserve-controller-manager-856948b99f-vjsc4" May 27 21:37:25.944311 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:25.941427 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/5b9e9a6e-a950-496c-937a-f6563ee83fc5-operator-config\") pod \"servicemesh-operator3-55f49c5f94-vzw72\" (UID: \"5b9e9a6e-a950-496c-937a-f6563ee83fc5\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-vzw72" May 27 21:37:25.944311 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:37:25.941575 2564 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found May 27 21:37:25.944311 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:37:25.941657 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ef19ce3-6eb8-4064-9c5e-31eb60b8c56a-cert podName:7ef19ce3-6eb8-4064-9c5e-31eb60b8c56a nodeName:}" failed. No retries permitted until 2026-05-27 21:37:26.441638736 +0000 UTC m=+519.915843445 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7ef19ce3-6eb8-4064-9c5e-31eb60b8c56a-cert") pod "kserve-controller-manager-856948b99f-vjsc4" (UID: "7ef19ce3-6eb8-4064-9c5e-31eb60b8c56a") : secret "kserve-webhook-server-cert" not found May 27 21:37:25.966665 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:25.966638 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wft4k\" (UniqueName: \"kubernetes.io/projected/7ef19ce3-6eb8-4064-9c5e-31eb60b8c56a-kube-api-access-wft4k\") pod \"kserve-controller-manager-856948b99f-vjsc4\" (UID: \"7ef19ce3-6eb8-4064-9c5e-31eb60b8c56a\") " pod="opendatahub/kserve-controller-manager-856948b99f-vjsc4" May 27 21:37:26.042107 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:26.042067 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-c4lnw\" (UniqueName: \"kubernetes.io/projected/5b9e9a6e-a950-496c-937a-f6563ee83fc5-kube-api-access-c4lnw\") pod \"servicemesh-operator3-55f49c5f94-vzw72\" (UID: \"5b9e9a6e-a950-496c-937a-f6563ee83fc5\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-vzw72" May 27 21:37:26.042332 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:26.042161 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/5b9e9a6e-a950-496c-937a-f6563ee83fc5-operator-config\") pod \"servicemesh-operator3-55f49c5f94-vzw72\" (UID: \"5b9e9a6e-a950-496c-937a-f6563ee83fc5\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-vzw72" May 27 21:37:26.045486 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:26.045453 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/5b9e9a6e-a950-496c-937a-f6563ee83fc5-operator-config\") pod \"servicemesh-operator3-55f49c5f94-vzw72\" (UID: \"5b9e9a6e-a950-496c-937a-f6563ee83fc5\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-vzw72" May 27 21:37:26.057632 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:26.057604 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4lnw\" (UniqueName: \"kubernetes.io/projected/5b9e9a6e-a950-496c-937a-f6563ee83fc5-kube-api-access-c4lnw\") pod \"servicemesh-operator3-55f49c5f94-vzw72\" (UID: \"5b9e9a6e-a950-496c-937a-f6563ee83fc5\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-vzw72" May 27 21:37:26.130688 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:26.130650 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-55f49c5f94-vzw72" May 27 21:37:26.338300 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:26.338252 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-vzw72"] May 27 21:37:26.339476 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:37:26.339449 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b9e9a6e_a950_496c_937a_f6563ee83fc5.slice/crio-4ae9c4189569f636cc4af3045851153f174733d8b849baaef4c95aa662c76319 WatchSource:0}: Error finding container 4ae9c4189569f636cc4af3045851153f174733d8b849baaef4c95aa662c76319: Status 404 returned error can't find the container with id 4ae9c4189569f636cc4af3045851153f174733d8b849baaef4c95aa662c76319 May 27 21:37:26.448146 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:26.448042 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7ef19ce3-6eb8-4064-9c5e-31eb60b8c56a-cert\") pod \"kserve-controller-manager-856948b99f-vjsc4\" (UID: \"7ef19ce3-6eb8-4064-9c5e-31eb60b8c56a\") " pod="opendatahub/kserve-controller-manager-856948b99f-vjsc4" May 27 21:37:26.451227 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:26.451202 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7ef19ce3-6eb8-4064-9c5e-31eb60b8c56a-cert\") pod \"kserve-controller-manager-856948b99f-vjsc4\" (UID: \"7ef19ce3-6eb8-4064-9c5e-31eb60b8c56a\") " pod="opendatahub/kserve-controller-manager-856948b99f-vjsc4" May 27 21:37:26.575476 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:26.575438 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-856948b99f-vjsc4" May 27 21:37:26.706587 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:26.706560 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-856948b99f-vjsc4"] May 27 21:37:26.708319 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:37:26.708290 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ef19ce3_6eb8_4064_9c5e_31eb60b8c56a.slice/crio-0ed16b551672e5390777f99fe11d7f1b87df05ef4eb29eba76f17a7370348266 WatchSource:0}: Error finding container 0ed16b551672e5390777f99fe11d7f1b87df05ef4eb29eba76f17a7370348266: Status 404 returned error can't find the container with id 0ed16b551672e5390777f99fe11d7f1b87df05ef4eb29eba76f17a7370348266 May 27 21:37:26.915590 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:26.915555 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-55f49c5f94-vzw72" event={"ID":"5b9e9a6e-a950-496c-937a-f6563ee83fc5","Type":"ContainerStarted","Data":"4ae9c4189569f636cc4af3045851153f174733d8b849baaef4c95aa662c76319"} May 27 21:37:26.916607 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:26.916584 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-856948b99f-vjsc4" event={"ID":"7ef19ce3-6eb8-4064-9c5e-31eb60b8c56a","Type":"ContainerStarted","Data":"0ed16b551672e5390777f99fe11d7f1b87df05ef4eb29eba76f17a7370348266"} May 27 21:37:26.918344 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:26.918320 2564 generic.go:358] "Generic (PLEG): container finished" podID="64699393-17ae-460f-ab95-a1f75fc2d7e7" containerID="dbe6d7d578f5767ceb670a661d0458dc19f516511258de9b3a0c1cab495c8dbc" exitCode=0 May 27 21:37:26.918441 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:26.918375 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a8c5h8" event={"ID":"64699393-17ae-460f-ab95-a1f75fc2d7e7","Type":"ContainerDied","Data":"dbe6d7d578f5767ceb670a661d0458dc19f516511258de9b3a0c1cab495c8dbc"} May 27 21:37:27.505886 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:27.505844 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-858dbf95b8-tn94s" May 27 21:37:27.506446 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:27.506425 2564 scope.go:117] "RemoveContainer" containerID="18ac0ee028321c751f9e80d72eeab36bb660ea77afd284c563bb8437be5c592d" May 27 21:37:27.506696 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:37:27.506674 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=odh-model-controller-858dbf95b8-tn94s_opendatahub(6333b9d0-8455-4556-adde-669e2259a722)\"" pod="opendatahub/odh-model-controller-858dbf95b8-tn94s" podUID="6333b9d0-8455-4556-adde-669e2259a722" May 27 21:37:29.023413 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:29.023384 2564 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a8c5h8" May 27 21:37:29.075826 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:29.075798 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/64699393-17ae-460f-ab95-a1f75fc2d7e7-bundle\") pod \"64699393-17ae-460f-ab95-a1f75fc2d7e7\" (UID: \"64699393-17ae-460f-ab95-a1f75fc2d7e7\") " May 27 21:37:29.075951 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:29.075854 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wmqjs\" (UniqueName: \"kubernetes.io/projected/64699393-17ae-460f-ab95-a1f75fc2d7e7-kube-api-access-wmqjs\") pod \"64699393-17ae-460f-ab95-a1f75fc2d7e7\" (UID: \"64699393-17ae-460f-ab95-a1f75fc2d7e7\") " May 27 21:37:29.075951 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:29.075936 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/64699393-17ae-460f-ab95-a1f75fc2d7e7-util\") pod \"64699393-17ae-460f-ab95-a1f75fc2d7e7\" (UID: \"64699393-17ae-460f-ab95-a1f75fc2d7e7\") " May 27 21:37:29.077189 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:29.077122 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64699393-17ae-460f-ab95-a1f75fc2d7e7-bundle" (OuterVolumeSpecName: "bundle") pod "64699393-17ae-460f-ab95-a1f75fc2d7e7" (UID: "64699393-17ae-460f-ab95-a1f75fc2d7e7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 27 21:37:29.078362 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:29.078333 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64699393-17ae-460f-ab95-a1f75fc2d7e7-kube-api-access-wmqjs" (OuterVolumeSpecName: "kube-api-access-wmqjs") pod "64699393-17ae-460f-ab95-a1f75fc2d7e7" (UID: "64699393-17ae-460f-ab95-a1f75fc2d7e7"). InnerVolumeSpecName "kube-api-access-wmqjs". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 27 21:37:29.080516 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:29.080495 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64699393-17ae-460f-ab95-a1f75fc2d7e7-util" (OuterVolumeSpecName: "util") pod "64699393-17ae-460f-ab95-a1f75fc2d7e7" (UID: "64699393-17ae-460f-ab95-a1f75fc2d7e7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 27 21:37:29.176548 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:29.176515 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/64699393-17ae-460f-ab95-a1f75fc2d7e7-bundle\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:37:29.176548 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:29.176545 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wmqjs\" (UniqueName: \"kubernetes.io/projected/64699393-17ae-460f-ab95-a1f75fc2d7e7-kube-api-access-wmqjs\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:37:29.176548 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:29.176556 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/64699393-17ae-460f-ab95-a1f75fc2d7e7-util\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:37:29.933595 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:29.933543 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a8c5h8" event={"ID":"64699393-17ae-460f-ab95-a1f75fc2d7e7","Type":"ContainerDied","Data":"bf7c1827279748f7b5013c7e07758e6dab5440ce7d41c8c78a1506ec765ff2c0"} May 27 21:37:29.933595 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:29.933591 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bf7c1827279748f7b5013c7e07758e6dab5440ce7d41c8c78a1506ec765ff2c0" May 27 21:37:29.933595 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:29.933559 2564 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a8c5h8" May 27 21:37:29.935870 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:29.935835 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-55f49c5f94-vzw72" event={"ID":"5b9e9a6e-a950-496c-937a-f6563ee83fc5","Type":"ContainerStarted","Data":"1818bcea51d61164e5f7d1827ba4afd4737c603706ccf4bf818b173c633ed02b"} May 27 21:37:29.936059 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:29.936006 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-operators/servicemesh-operator3-55f49c5f94-vzw72" May 27 21:37:29.965553 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:29.965487 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-operators/servicemesh-operator3-55f49c5f94-vzw72" podStartSLOduration=2.253355543 podStartE2EDuration="4.965466386s" podCreationTimestamp="2026-05-27 21:37:25 +0000 UTC" firstStartedPulling="2026-05-27 21:37:26.34211384 +0000 UTC m=+519.816318548" lastFinishedPulling="2026-05-27 21:37:29.054224667 +0000 UTC m=+522.528429391" observedRunningTime="2026-05-27 21:37:29.961777086 +0000 UTC m=+523.435981830" watchObservedRunningTime="2026-05-27 21:37:29.965466386 +0000 UTC m=+523.439671118" May 27 21:37:30.940632 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:30.940584 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-856948b99f-vjsc4" event={"ID":"7ef19ce3-6eb8-4064-9c5e-31eb60b8c56a","Type":"ContainerStarted","Data":"a273dd75ec7b9f30defae71015abc7d7b9900f92fe137b9621c6492824ae9ca6"} May 27 21:37:30.941071 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:30.940878 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-856948b99f-vjsc4" May 27 21:37:30.986888 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:30.986836 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-856948b99f-vjsc4" podStartSLOduration=2.665974471 podStartE2EDuration="5.986822193s" podCreationTimestamp="2026-05-27 21:37:25 +0000 UTC" firstStartedPulling="2026-05-27 21:37:26.709654674 +0000 UTC m=+520.183859381" lastFinishedPulling="2026-05-27 21:37:30.030502383 +0000 UTC m=+523.504707103" observedRunningTime="2026-05-27 21:37:30.984033795 +0000 UTC m=+524.458238525" watchObservedRunningTime="2026-05-27 21:37:30.986822193 +0000 UTC m=+524.461026923" May 27 21:37:35.188073 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:35.188031 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-pbr96"] May 27 21:37:35.188542 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:35.188524 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64699393-17ae-460f-ab95-a1f75fc2d7e7" containerName="extract" May 27 21:37:35.188605 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:35.188545 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="64699393-17ae-460f-ab95-a1f75fc2d7e7" containerName="extract" May 27 21:37:35.188605 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:35.188558 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64699393-17ae-460f-ab95-a1f75fc2d7e7" containerName="util" May 27 21:37:35.188605 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:35.188568 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="64699393-17ae-460f-ab95-a1f75fc2d7e7" containerName="util" May 27 21:37:35.188605 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:35.188582 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64699393-17ae-460f-ab95-a1f75fc2d7e7" containerName="pull" May 27 21:37:35.188605 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:35.188591 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="64699393-17ae-460f-ab95-a1f75fc2d7e7" containerName="pull" May 27 21:37:35.188751 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:35.188669 2564 memory_manager.go:356] "RemoveStaleState removing state" podUID="64699393-17ae-460f-ab95-a1f75fc2d7e7" containerName="extract" May 27 21:37:35.192136 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:35.192111 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-pbr96" May 27 21:37:35.194863 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:35.194841 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" May 27 21:37:35.196802 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:35.196778 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" May 27 21:37:35.196802 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:35.196800 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-fbg8q\"" May 27 21:37:35.196991 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:35.196788 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" May 27 21:37:35.196991 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:35.196788 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" May 27 21:37:35.202570 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:35.202549 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-pbr96"] May 27 21:37:35.228143 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:35.228114 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b04535f7-68a3-4874-af75-5ca190852e78-istio-token\") pod \"istiod-openshift-gateway-798958bb55-pbr96\" (UID: \"b04535f7-68a3-4874-af75-5ca190852e78\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-pbr96" May 27 21:37:35.228306 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:35.228149 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b04535f7-68a3-4874-af75-5ca190852e78-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-pbr96\" (UID: \"b04535f7-68a3-4874-af75-5ca190852e78\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-pbr96" May 27 21:37:35.228306 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:35.228181 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjq98\" (UniqueName: \"kubernetes.io/projected/b04535f7-68a3-4874-af75-5ca190852e78-kube-api-access-qjq98\") pod \"istiod-openshift-gateway-798958bb55-pbr96\" (UID: \"b04535f7-68a3-4874-af75-5ca190852e78\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-pbr96" May 27 21:37:35.228306 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:35.228285 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b04535f7-68a3-4874-af75-5ca190852e78-local-certs\") pod \"istiod-openshift-gateway-798958bb55-pbr96\" (UID: \"b04535f7-68a3-4874-af75-5ca190852e78\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-pbr96" May 27 21:37:35.228474 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:35.228314 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b04535f7-68a3-4874-af75-5ca190852e78-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-pbr96\" (UID: \"b04535f7-68a3-4874-af75-5ca190852e78\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-pbr96" May 27 21:37:35.228474 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:35.228348 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b04535f7-68a3-4874-af75-5ca190852e78-cacerts\") pod \"istiod-openshift-gateway-798958bb55-pbr96\" (UID: \"b04535f7-68a3-4874-af75-5ca190852e78\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-pbr96" May 27 21:37:35.228474 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:35.228407 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b04535f7-68a3-4874-af75-5ca190852e78-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-pbr96\" (UID: \"b04535f7-68a3-4874-af75-5ca190852e78\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-pbr96" May 27 21:37:35.329463 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:35.329381 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b04535f7-68a3-4874-af75-5ca190852e78-local-certs\") pod \"istiod-openshift-gateway-798958bb55-pbr96\" (UID: \"b04535f7-68a3-4874-af75-5ca190852e78\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-pbr96" May 27 21:37:35.329463 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:35.329421 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b04535f7-68a3-4874-af75-5ca190852e78-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-pbr96\" (UID: \"b04535f7-68a3-4874-af75-5ca190852e78\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-pbr96" May 27 21:37:35.329463 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:35.329451 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b04535f7-68a3-4874-af75-5ca190852e78-cacerts\") pod \"istiod-openshift-gateway-798958bb55-pbr96\" (UID: \"b04535f7-68a3-4874-af75-5ca190852e78\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-pbr96" May 27 21:37:35.329751 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:35.329469 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b04535f7-68a3-4874-af75-5ca190852e78-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-pbr96\" (UID: \"b04535f7-68a3-4874-af75-5ca190852e78\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-pbr96" May 27 21:37:35.329751 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:35.329498 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b04535f7-68a3-4874-af75-5ca190852e78-istio-token\") pod \"istiod-openshift-gateway-798958bb55-pbr96\" (UID: \"b04535f7-68a3-4874-af75-5ca190852e78\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-pbr96" May 27 21:37:35.329751 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:35.329513 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b04535f7-68a3-4874-af75-5ca190852e78-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-pbr96\" (UID: \"b04535f7-68a3-4874-af75-5ca190852e78\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-pbr96" May 27 21:37:35.329751 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:35.329539 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qjq98\" (UniqueName: \"kubernetes.io/projected/b04535f7-68a3-4874-af75-5ca190852e78-kube-api-access-qjq98\") pod \"istiod-openshift-gateway-798958bb55-pbr96\" (UID: \"b04535f7-68a3-4874-af75-5ca190852e78\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-pbr96" May 27 21:37:35.330170 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:35.330143 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b04535f7-68a3-4874-af75-5ca190852e78-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-pbr96\" (UID: \"b04535f7-68a3-4874-af75-5ca190852e78\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-pbr96" May 27 21:37:35.332377 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:35.332352 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b04535f7-68a3-4874-af75-5ca190852e78-cacerts\") pod \"istiod-openshift-gateway-798958bb55-pbr96\" (UID: \"b04535f7-68a3-4874-af75-5ca190852e78\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-pbr96" May 27 21:37:35.332466 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:35.332404 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b04535f7-68a3-4874-af75-5ca190852e78-local-certs\") pod \"istiod-openshift-gateway-798958bb55-pbr96\" (UID: \"b04535f7-68a3-4874-af75-5ca190852e78\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-pbr96" May 27 21:37:35.332506 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:35.332488 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b04535f7-68a3-4874-af75-5ca190852e78-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-pbr96\" (UID: \"b04535f7-68a3-4874-af75-5ca190852e78\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-pbr96" May 27 21:37:35.332541 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:35.332499 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b04535f7-68a3-4874-af75-5ca190852e78-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-pbr96\" (UID: \"b04535f7-68a3-4874-af75-5ca190852e78\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-pbr96" May 27 21:37:35.338137 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:35.338087 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b04535f7-68a3-4874-af75-5ca190852e78-istio-token\") pod \"istiod-openshift-gateway-798958bb55-pbr96\" (UID: \"b04535f7-68a3-4874-af75-5ca190852e78\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-pbr96" May 27 21:37:35.338371 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:35.338356 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjq98\" (UniqueName: \"kubernetes.io/projected/b04535f7-68a3-4874-af75-5ca190852e78-kube-api-access-qjq98\") pod \"istiod-openshift-gateway-798958bb55-pbr96\" (UID: \"b04535f7-68a3-4874-af75-5ca190852e78\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-pbr96" May 27 21:37:35.501871 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:35.501774 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-pbr96" May 27 21:37:35.639642 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:35.639611 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-pbr96"] May 27 21:37:35.641209 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:37:35.641178 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb04535f7_68a3_4874_af75_5ca190852e78.slice/crio-1e85ceb94027bde672ee344a0c439328a7d0d7e2dc5004086849cd94f912ae1c WatchSource:0}: Error finding container 1e85ceb94027bde672ee344a0c439328a7d0d7e2dc5004086849cd94f912ae1c: Status 404 returned error can't find the container with id 1e85ceb94027bde672ee344a0c439328a7d0d7e2dc5004086849cd94f912ae1c May 27 21:37:35.960234 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:35.960201 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-pbr96" event={"ID":"b04535f7-68a3-4874-af75-5ca190852e78","Type":"ContainerStarted","Data":"1e85ceb94027bde672ee344a0c439328a7d0d7e2dc5004086849cd94f912ae1c"} May 27 21:37:37.505697 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:37.505660 2564 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="opendatahub/odh-model-controller-858dbf95b8-tn94s" May 27 21:37:37.506117 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:37.506101 2564 scope.go:117] "RemoveContainer" containerID="18ac0ee028321c751f9e80d72eeab36bb660ea77afd284c563bb8437be5c592d" May 27 21:37:38.174880 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:38.174839 2564 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} May 27 21:37:38.174971 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:38.174908 2564 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} May 27 21:37:38.974655 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:38.974615 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-pbr96" event={"ID":"b04535f7-68a3-4874-af75-5ca190852e78","Type":"ContainerStarted","Data":"7cf7a29a9de8ba9d343f925f4f5d2a33c2a2f79640b5898a5d8c284ea325c415"} May 27 21:37:38.975129 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:38.974699 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-pbr96" May 27 21:37:38.976572 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:38.976531 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-tn94s" event={"ID":"6333b9d0-8455-4556-adde-669e2259a722","Type":"ContainerStarted","Data":"cbace93548f2d1f3c084f98b4f25c300e16738f00c3503ade19a6c4879825213"} May 27 21:37:38.976734 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:38.976720 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-858dbf95b8-tn94s" May 27 21:37:39.002471 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:39.002414 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-pbr96" podStartSLOduration=1.471309081 podStartE2EDuration="4.002398881s" podCreationTimestamp="2026-05-27 21:37:35 +0000 UTC" firstStartedPulling="2026-05-27 21:37:35.643501822 +0000 UTC m=+529.117706533" lastFinishedPulling="2026-05-27 21:37:38.174591621 +0000 UTC m=+531.648796333" observedRunningTime="2026-05-27 21:37:38.997195995 +0000 UTC m=+532.471400717" watchObservedRunningTime="2026-05-27 21:37:39.002398881 +0000 UTC m=+532.476603603" May 27 21:37:39.018372 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:39.018324 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/odh-model-controller-858dbf95b8-tn94s" podStartSLOduration=2.960452439 podStartE2EDuration="23.01831091s" podCreationTimestamp="2026-05-27 21:37:16 +0000 UTC" firstStartedPulling="2026-05-27 21:37:18.116909145 +0000 UTC m=+511.591113857" lastFinishedPulling="2026-05-27 21:37:38.174767601 +0000 UTC m=+531.648972328" observedRunningTime="2026-05-27 21:37:39.016457834 +0000 UTC m=+532.490662565" watchObservedRunningTime="2026-05-27 21:37:39.01831091 +0000 UTC m=+532.492515640" May 27 21:37:39.982208 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:39.982180 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-pbr96" May 27 21:37:40.943178 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:40.943137 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-operators/servicemesh-operator3-55f49c5f94-vzw72" May 27 21:37:41.163402 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:41.163366 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-65fcc74dd8-6wspm"] May 27 21:37:41.168858 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:41.168834 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65fcc74dd8-6wspm" May 27 21:37:41.182855 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:41.182822 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-65fcc74dd8-6wspm"] May 27 21:37:41.289554 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:41.289459 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/038f205d-8fa0-4849-a4f3-bfef23617a86-console-oauth-config\") pod \"console-65fcc74dd8-6wspm\" (UID: \"038f205d-8fa0-4849-a4f3-bfef23617a86\") " pod="openshift-console/console-65fcc74dd8-6wspm" May 27 21:37:41.289725 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:41.289606 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/038f205d-8fa0-4849-a4f3-bfef23617a86-console-serving-cert\") pod \"console-65fcc74dd8-6wspm\" (UID: \"038f205d-8fa0-4849-a4f3-bfef23617a86\") " pod="openshift-console/console-65fcc74dd8-6wspm" May 27 21:37:41.289725 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:41.289700 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/038f205d-8fa0-4849-a4f3-bfef23617a86-service-ca\") pod \"console-65fcc74dd8-6wspm\" (UID: \"038f205d-8fa0-4849-a4f3-bfef23617a86\") " pod="openshift-console/console-65fcc74dd8-6wspm" May 27 21:37:41.289855 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:41.289762 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txxrd\" (UniqueName: \"kubernetes.io/projected/038f205d-8fa0-4849-a4f3-bfef23617a86-kube-api-access-txxrd\") pod \"console-65fcc74dd8-6wspm\" (UID: \"038f205d-8fa0-4849-a4f3-bfef23617a86\") " pod="openshift-console/console-65fcc74dd8-6wspm" May 27 21:37:41.289855 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:41.289826 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/038f205d-8fa0-4849-a4f3-bfef23617a86-trusted-ca-bundle\") pod \"console-65fcc74dd8-6wspm\" (UID: \"038f205d-8fa0-4849-a4f3-bfef23617a86\") " pod="openshift-console/console-65fcc74dd8-6wspm" May 27 21:37:41.289955 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:41.289881 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/038f205d-8fa0-4849-a4f3-bfef23617a86-console-config\") pod \"console-65fcc74dd8-6wspm\" (UID: \"038f205d-8fa0-4849-a4f3-bfef23617a86\") " pod="openshift-console/console-65fcc74dd8-6wspm" May 27 21:37:41.290006 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:41.289954 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/038f205d-8fa0-4849-a4f3-bfef23617a86-oauth-serving-cert\") pod \"console-65fcc74dd8-6wspm\" (UID: \"038f205d-8fa0-4849-a4f3-bfef23617a86\") " pod="openshift-console/console-65fcc74dd8-6wspm" May 27 21:37:41.390536 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:41.390500 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/038f205d-8fa0-4849-a4f3-bfef23617a86-service-ca\") pod \"console-65fcc74dd8-6wspm\" (UID: \"038f205d-8fa0-4849-a4f3-bfef23617a86\") " pod="openshift-console/console-65fcc74dd8-6wspm" May 27 21:37:41.390536 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:41.390542 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-txxrd\" (UniqueName: \"kubernetes.io/projected/038f205d-8fa0-4849-a4f3-bfef23617a86-kube-api-access-txxrd\") pod \"console-65fcc74dd8-6wspm\" (UID: \"038f205d-8fa0-4849-a4f3-bfef23617a86\") " pod="openshift-console/console-65fcc74dd8-6wspm" May 27 21:37:41.390797 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:41.390571 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/038f205d-8fa0-4849-a4f3-bfef23617a86-trusted-ca-bundle\") pod \"console-65fcc74dd8-6wspm\" (UID: \"038f205d-8fa0-4849-a4f3-bfef23617a86\") " pod="openshift-console/console-65fcc74dd8-6wspm" May 27 21:37:41.390797 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:41.390599 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/038f205d-8fa0-4849-a4f3-bfef23617a86-console-config\") pod \"console-65fcc74dd8-6wspm\" (UID: \"038f205d-8fa0-4849-a4f3-bfef23617a86\") " pod="openshift-console/console-65fcc74dd8-6wspm" May 27 21:37:41.390797 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:41.390626 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/038f205d-8fa0-4849-a4f3-bfef23617a86-oauth-serving-cert\") pod \"console-65fcc74dd8-6wspm\" (UID: \"038f205d-8fa0-4849-a4f3-bfef23617a86\") " pod="openshift-console/console-65fcc74dd8-6wspm" May 27 21:37:41.390797 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:41.390654 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/038f205d-8fa0-4849-a4f3-bfef23617a86-console-oauth-config\") pod \"console-65fcc74dd8-6wspm\" (UID: \"038f205d-8fa0-4849-a4f3-bfef23617a86\") " pod="openshift-console/console-65fcc74dd8-6wspm" May 27 21:37:41.390797 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:41.390684 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/038f205d-8fa0-4849-a4f3-bfef23617a86-console-serving-cert\") pod \"console-65fcc74dd8-6wspm\" (UID: \"038f205d-8fa0-4849-a4f3-bfef23617a86\") " pod="openshift-console/console-65fcc74dd8-6wspm" May 27 21:37:41.391333 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:41.391304 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/038f205d-8fa0-4849-a4f3-bfef23617a86-service-ca\") pod \"console-65fcc74dd8-6wspm\" (UID: \"038f205d-8fa0-4849-a4f3-bfef23617a86\") " pod="openshift-console/console-65fcc74dd8-6wspm" May 27 21:37:41.391462 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:41.391427 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/038f205d-8fa0-4849-a4f3-bfef23617a86-oauth-serving-cert\") pod \"console-65fcc74dd8-6wspm\" (UID: \"038f205d-8fa0-4849-a4f3-bfef23617a86\") " pod="openshift-console/console-65fcc74dd8-6wspm" May 27 21:37:41.391506 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:41.391461 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/038f205d-8fa0-4849-a4f3-bfef23617a86-console-config\") pod \"console-65fcc74dd8-6wspm\" (UID: \"038f205d-8fa0-4849-a4f3-bfef23617a86\") " pod="openshift-console/console-65fcc74dd8-6wspm" May 27 21:37:41.391571 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:41.391549 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/038f205d-8fa0-4849-a4f3-bfef23617a86-trusted-ca-bundle\") pod \"console-65fcc74dd8-6wspm\" (UID: \"038f205d-8fa0-4849-a4f3-bfef23617a86\") " pod="openshift-console/console-65fcc74dd8-6wspm" May 27 21:37:41.393123 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:41.393103 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/038f205d-8fa0-4849-a4f3-bfef23617a86-console-oauth-config\") pod \"console-65fcc74dd8-6wspm\" (UID: \"038f205d-8fa0-4849-a4f3-bfef23617a86\") " pod="openshift-console/console-65fcc74dd8-6wspm" May 27 21:37:41.393345 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:41.393326 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/038f205d-8fa0-4849-a4f3-bfef23617a86-console-serving-cert\") pod \"console-65fcc74dd8-6wspm\" (UID: \"038f205d-8fa0-4849-a4f3-bfef23617a86\") " pod="openshift-console/console-65fcc74dd8-6wspm" May 27 21:37:41.419011 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:41.418988 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-txxrd\" (UniqueName: \"kubernetes.io/projected/038f205d-8fa0-4849-a4f3-bfef23617a86-kube-api-access-txxrd\") pod \"console-65fcc74dd8-6wspm\" (UID: \"038f205d-8fa0-4849-a4f3-bfef23617a86\") " pod="openshift-console/console-65fcc74dd8-6wspm" May 27 21:37:41.481103 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:41.481073 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65fcc74dd8-6wspm" May 27 21:37:41.634956 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:41.634927 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-65fcc74dd8-6wspm"] May 27 21:37:41.636791 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:37:41.636750 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod038f205d_8fa0_4849_a4f3_bfef23617a86.slice/crio-36ca4eb7536ff74d546771d40229234dac937387a08b9ca8b4392d871492408c WatchSource:0}: Error finding container 36ca4eb7536ff74d546771d40229234dac937387a08b9ca8b4392d871492408c: Status 404 returned error can't find the container with id 36ca4eb7536ff74d546771d40229234dac937387a08b9ca8b4392d871492408c May 27 21:37:41.991332 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:41.991292 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65fcc74dd8-6wspm" event={"ID":"038f205d-8fa0-4849-a4f3-bfef23617a86","Type":"ContainerStarted","Data":"9a1c692abc0e853c891e3944214087b532488b7ab0f569fedb24ff158a366a69"} May 27 21:37:41.991332 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:41.991335 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65fcc74dd8-6wspm" event={"ID":"038f205d-8fa0-4849-a4f3-bfef23617a86","Type":"ContainerStarted","Data":"36ca4eb7536ff74d546771d40229234dac937387a08b9ca8b4392d871492408c"} May 27 21:37:42.060099 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:42.060041 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-65fcc74dd8-6wspm" podStartSLOduration=1.060020486 podStartE2EDuration="1.060020486s" podCreationTimestamp="2026-05-27 21:37:41 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-27 21:37:42.05999133 +0000 UTC m=+535.534196060" watchObservedRunningTime="2026-05-27 21:37:42.060020486 +0000 UTC m=+535.534225217" May 27 21:37:49.982941 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:49.982897 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-858dbf95b8-tn94s" May 27 21:37:51.481571 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:51.481536 2564 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-65fcc74dd8-6wspm" May 27 21:37:51.481571 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:51.481570 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-65fcc74dd8-6wspm" May 27 21:37:51.486305 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:51.486283 2564 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-65fcc74dd8-6wspm" May 27 21:37:52.030609 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:52.030581 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-65fcc74dd8-6wspm" May 27 21:37:52.083751 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:37:52.083713 2564 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77fdcbc69d-wvxxh"] May 27 21:38:01.950178 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:01.950145 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-856948b99f-vjsc4" May 27 21:38:17.107872 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:17.107805 2564 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-77fdcbc69d-wvxxh" podUID="b520c3be-d3f3-4cb2-a739-444a8a5975ba" containerName="console" containerID="cri-o://09a6651f670b9534a33330c162420ab38910952b6d70f720428d38680f1df663" gracePeriod=15 May 27 21:38:17.358978 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:17.358916 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-77fdcbc69d-wvxxh_b520c3be-d3f3-4cb2-a739-444a8a5975ba/console/0.log" May 27 21:38:17.359092 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:17.358986 2564 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77fdcbc69d-wvxxh" May 27 21:38:17.500158 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:17.500125 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b520c3be-d3f3-4cb2-a739-444a8a5975ba-console-oauth-config\") pod \"b520c3be-d3f3-4cb2-a739-444a8a5975ba\" (UID: \"b520c3be-d3f3-4cb2-a739-444a8a5975ba\") " May 27 21:38:17.500345 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:17.500201 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b520c3be-d3f3-4cb2-a739-444a8a5975ba-service-ca\") pod \"b520c3be-d3f3-4cb2-a739-444a8a5975ba\" (UID: \"b520c3be-d3f3-4cb2-a739-444a8a5975ba\") " May 27 21:38:17.500345 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:17.500228 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b520c3be-d3f3-4cb2-a739-444a8a5975ba-console-config\") pod \"b520c3be-d3f3-4cb2-a739-444a8a5975ba\" (UID: \"b520c3be-d3f3-4cb2-a739-444a8a5975ba\") " May 27 21:38:17.500345 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:17.500250 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b520c3be-d3f3-4cb2-a739-444a8a5975ba-trusted-ca-bundle\") pod \"b520c3be-d3f3-4cb2-a739-444a8a5975ba\" (UID: \"b520c3be-d3f3-4cb2-a739-444a8a5975ba\") " May 27 21:38:17.500345 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:17.500297 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b520c3be-d3f3-4cb2-a739-444a8a5975ba-oauth-serving-cert\") pod \"b520c3be-d3f3-4cb2-a739-444a8a5975ba\" (UID: \"b520c3be-d3f3-4cb2-a739-444a8a5975ba\") " May 27 21:38:17.500552 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:17.500388 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b520c3be-d3f3-4cb2-a739-444a8a5975ba-console-serving-cert\") pod \"b520c3be-d3f3-4cb2-a739-444a8a5975ba\" (UID: \"b520c3be-d3f3-4cb2-a739-444a8a5975ba\") " May 27 21:38:17.500552 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:17.500414 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tkxnp\" (UniqueName: \"kubernetes.io/projected/b520c3be-d3f3-4cb2-a739-444a8a5975ba-kube-api-access-tkxnp\") pod \"b520c3be-d3f3-4cb2-a739-444a8a5975ba\" (UID: \"b520c3be-d3f3-4cb2-a739-444a8a5975ba\") " May 27 21:38:17.500734 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:17.500624 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b520c3be-d3f3-4cb2-a739-444a8a5975ba-service-ca" (OuterVolumeSpecName: "service-ca") pod "b520c3be-d3f3-4cb2-a739-444a8a5975ba" (UID: "b520c3be-d3f3-4cb2-a739-444a8a5975ba"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 27 21:38:17.500734 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:17.500685 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b520c3be-d3f3-4cb2-a739-444a8a5975ba-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "b520c3be-d3f3-4cb2-a739-444a8a5975ba" (UID: "b520c3be-d3f3-4cb2-a739-444a8a5975ba"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 27 21:38:17.501052 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:17.500729 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b520c3be-d3f3-4cb2-a739-444a8a5975ba-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b520c3be-d3f3-4cb2-a739-444a8a5975ba" (UID: "b520c3be-d3f3-4cb2-a739-444a8a5975ba"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 27 21:38:17.501052 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:17.500898 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b520c3be-d3f3-4cb2-a739-444a8a5975ba-console-config" (OuterVolumeSpecName: "console-config") pod "b520c3be-d3f3-4cb2-a739-444a8a5975ba" (UID: "b520c3be-d3f3-4cb2-a739-444a8a5975ba"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 27 21:38:17.502382 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:17.502355 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b520c3be-d3f3-4cb2-a739-444a8a5975ba-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b520c3be-d3f3-4cb2-a739-444a8a5975ba" (UID: "b520c3be-d3f3-4cb2-a739-444a8a5975ba"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 27 21:38:17.502666 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:17.502640 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b520c3be-d3f3-4cb2-a739-444a8a5975ba-kube-api-access-tkxnp" (OuterVolumeSpecName: "kube-api-access-tkxnp") pod "b520c3be-d3f3-4cb2-a739-444a8a5975ba" (UID: "b520c3be-d3f3-4cb2-a739-444a8a5975ba"). InnerVolumeSpecName "kube-api-access-tkxnp". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 27 21:38:17.502791 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:17.502776 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b520c3be-d3f3-4cb2-a739-444a8a5975ba-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b520c3be-d3f3-4cb2-a739-444a8a5975ba" (UID: "b520c3be-d3f3-4cb2-a739-444a8a5975ba"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 27 21:38:17.601469 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:17.601440 2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b520c3be-d3f3-4cb2-a739-444a8a5975ba-service-ca\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:38:17.601469 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:17.601464 2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b520c3be-d3f3-4cb2-a739-444a8a5975ba-console-config\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:38:17.601469 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:17.601473 2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b520c3be-d3f3-4cb2-a739-444a8a5975ba-trusted-ca-bundle\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:38:17.601656 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:17.601483 2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b520c3be-d3f3-4cb2-a739-444a8a5975ba-oauth-serving-cert\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:38:17.601656 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:17.601491 2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b520c3be-d3f3-4cb2-a739-444a8a5975ba-console-serving-cert\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:38:17.601656 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:17.601500 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tkxnp\" (UniqueName: \"kubernetes.io/projected/b520c3be-d3f3-4cb2-a739-444a8a5975ba-kube-api-access-tkxnp\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:38:17.601656 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:17.601510 2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b520c3be-d3f3-4cb2-a739-444a8a5975ba-console-oauth-config\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:38:18.122633 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:18.122607 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-77fdcbc69d-wvxxh_b520c3be-d3f3-4cb2-a739-444a8a5975ba/console/0.log" May 27 21:38:18.123069 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:18.122647 2564 generic.go:358] "Generic (PLEG): container finished" podID="b520c3be-d3f3-4cb2-a739-444a8a5975ba" containerID="09a6651f670b9534a33330c162420ab38910952b6d70f720428d38680f1df663" exitCode=2 May 27 21:38:18.123069 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:18.122680 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77fdcbc69d-wvxxh" event={"ID":"b520c3be-d3f3-4cb2-a739-444a8a5975ba","Type":"ContainerDied","Data":"09a6651f670b9534a33330c162420ab38910952b6d70f720428d38680f1df663"} May 27 21:38:18.123069 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:18.122722 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77fdcbc69d-wvxxh" event={"ID":"b520c3be-d3f3-4cb2-a739-444a8a5975ba","Type":"ContainerDied","Data":"94dfa220ba9c462016503c7d300379ea47e1a1e8fc661e0ede9e599ee6cec65e"} May 27 21:38:18.123069 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:18.122722 2564 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77fdcbc69d-wvxxh" May 27 21:38:18.123069 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:18.122738 2564 scope.go:117] "RemoveContainer" containerID="09a6651f670b9534a33330c162420ab38910952b6d70f720428d38680f1df663" May 27 21:38:18.131175 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:18.131154 2564 scope.go:117] "RemoveContainer" containerID="09a6651f670b9534a33330c162420ab38910952b6d70f720428d38680f1df663" May 27 21:38:18.131478 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:38:18.131456 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09a6651f670b9534a33330c162420ab38910952b6d70f720428d38680f1df663\": container with ID starting with 09a6651f670b9534a33330c162420ab38910952b6d70f720428d38680f1df663 not found: ID does not exist" containerID="09a6651f670b9534a33330c162420ab38910952b6d70f720428d38680f1df663" May 27 21:38:18.131539 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:18.131486 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09a6651f670b9534a33330c162420ab38910952b6d70f720428d38680f1df663"} err="failed to get container status \"09a6651f670b9534a33330c162420ab38910952b6d70f720428d38680f1df663\": rpc error: code = NotFound desc = could not find container \"09a6651f670b9534a33330c162420ab38910952b6d70f720428d38680f1df663\": container with ID starting with 09a6651f670b9534a33330c162420ab38910952b6d70f720428d38680f1df663 not found: ID does not exist" May 27 21:38:18.145092 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:18.145067 2564 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77fdcbc69d-wvxxh"] May 27 21:38:18.149135 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:18.149112 2564 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-77fdcbc69d-wvxxh"] May 27 21:38:19.084691 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:19.084660 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b520c3be-d3f3-4cb2-a739-444a8a5975ba" path="/var/lib/kubelet/pods/b520c3be-d3f3-4cb2-a739-444a8a5975ba/volumes" May 27 21:38:29.270574 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:29.270542 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tqm2b"] May 27 21:38:29.270983 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:29.270871 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b520c3be-d3f3-4cb2-a739-444a8a5975ba" containerName="console" May 27 21:38:29.270983 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:29.270881 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b520c3be-d3f3-4cb2-a739-444a8a5975ba" containerName="console" May 27 21:38:29.270983 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:29.270943 2564 memory_manager.go:356] "RemoveStaleState removing state" podUID="b520c3be-d3f3-4cb2-a739-444a8a5975ba" containerName="console" May 27 21:38:29.275510 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:29.275491 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tqm2b" May 27 21:38:29.278388 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:29.278364 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-zgkln\"" May 27 21:38:29.278517 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:29.278374 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" May 27 21:38:29.278517 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:29.278374 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" May 27 21:38:29.282138 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:29.282115 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tqm2b"] May 27 21:38:29.294614 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:29.294584 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8515de3f-c993-4bd8-bb41-91ec6d3bbf2b-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tqm2b\" (UID: \"8515de3f-c993-4bd8-bb41-91ec6d3bbf2b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tqm2b" May 27 21:38:29.294746 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:29.294622 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnd7t\" (UniqueName: \"kubernetes.io/projected/8515de3f-c993-4bd8-bb41-91ec6d3bbf2b-kube-api-access-tnd7t\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tqm2b\" (UID: \"8515de3f-c993-4bd8-bb41-91ec6d3bbf2b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tqm2b" May 27 21:38:29.294746 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:29.294659 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8515de3f-c993-4bd8-bb41-91ec6d3bbf2b-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tqm2b\" (UID: \"8515de3f-c993-4bd8-bb41-91ec6d3bbf2b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tqm2b" May 27 21:38:29.395185 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:29.395146 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8515de3f-c993-4bd8-bb41-91ec6d3bbf2b-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tqm2b\" (UID: \"8515de3f-c993-4bd8-bb41-91ec6d3bbf2b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tqm2b" May 27 21:38:29.395185 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:29.395186 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tnd7t\" (UniqueName: \"kubernetes.io/projected/8515de3f-c993-4bd8-bb41-91ec6d3bbf2b-kube-api-access-tnd7t\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tqm2b\" (UID: \"8515de3f-c993-4bd8-bb41-91ec6d3bbf2b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tqm2b" May 27 21:38:29.395457 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:29.395208 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8515de3f-c993-4bd8-bb41-91ec6d3bbf2b-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tqm2b\" (UID: \"8515de3f-c993-4bd8-bb41-91ec6d3bbf2b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tqm2b" May 27 21:38:29.395632 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:29.395611 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8515de3f-c993-4bd8-bb41-91ec6d3bbf2b-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tqm2b\" (UID: \"8515de3f-c993-4bd8-bb41-91ec6d3bbf2b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tqm2b" May 27 21:38:29.395699 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:29.395633 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8515de3f-c993-4bd8-bb41-91ec6d3bbf2b-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tqm2b\" (UID: \"8515de3f-c993-4bd8-bb41-91ec6d3bbf2b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tqm2b" May 27 21:38:29.405422 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:29.405397 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnd7t\" (UniqueName: \"kubernetes.io/projected/8515de3f-c993-4bd8-bb41-91ec6d3bbf2b-kube-api-access-tnd7t\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tqm2b\" (UID: \"8515de3f-c993-4bd8-bb41-91ec6d3bbf2b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tqm2b" May 27 21:38:29.585642 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:29.585558 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tqm2b" May 27 21:38:29.711148 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:29.711118 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tqm2b"] May 27 21:38:29.712357 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:38:29.712324 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8515de3f_c993_4bd8_bb41_91ec6d3bbf2b.slice/crio-f59c0b725dd21d870e2e445294faac0c9b42a5110707110d53d9bbdb8ea0c400 WatchSource:0}: Error finding container f59c0b725dd21d870e2e445294faac0c9b42a5110707110d53d9bbdb8ea0c400: Status 404 returned error can't find the container with id f59c0b725dd21d870e2e445294faac0c9b42a5110707110d53d9bbdb8ea0c400 May 27 21:38:29.871786 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:29.871719 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xvgn2"] May 27 21:38:29.875091 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:29.875071 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xvgn2" May 27 21:38:29.883662 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:29.883636 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xvgn2"] May 27 21:38:29.900098 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:29.900071 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5grv\" (UniqueName: \"kubernetes.io/projected/f1233a43-b5ac-45a9-96e7-56b99df116b7-kube-api-access-g5grv\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xvgn2\" (UID: \"f1233a43-b5ac-45a9-96e7-56b99df116b7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xvgn2" May 27 21:38:29.900224 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:29.900118 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f1233a43-b5ac-45a9-96e7-56b99df116b7-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xvgn2\" (UID: \"f1233a43-b5ac-45a9-96e7-56b99df116b7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xvgn2" May 27 21:38:29.900224 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:29.900179 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f1233a43-b5ac-45a9-96e7-56b99df116b7-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xvgn2\" (UID: \"f1233a43-b5ac-45a9-96e7-56b99df116b7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xvgn2" May 27 21:38:30.000999 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:30.000955 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-g5grv\" (UniqueName: \"kubernetes.io/projected/f1233a43-b5ac-45a9-96e7-56b99df116b7-kube-api-access-g5grv\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xvgn2\" (UID: \"f1233a43-b5ac-45a9-96e7-56b99df116b7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xvgn2" May 27 21:38:30.001176 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:30.001005 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f1233a43-b5ac-45a9-96e7-56b99df116b7-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xvgn2\" (UID: \"f1233a43-b5ac-45a9-96e7-56b99df116b7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xvgn2" May 27 21:38:30.001176 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:30.001063 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f1233a43-b5ac-45a9-96e7-56b99df116b7-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xvgn2\" (UID: \"f1233a43-b5ac-45a9-96e7-56b99df116b7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xvgn2" May 27 21:38:30.001415 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:30.001395 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f1233a43-b5ac-45a9-96e7-56b99df116b7-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xvgn2\" (UID: \"f1233a43-b5ac-45a9-96e7-56b99df116b7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xvgn2" May 27 21:38:30.001500 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:30.001463 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f1233a43-b5ac-45a9-96e7-56b99df116b7-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xvgn2\" (UID: \"f1233a43-b5ac-45a9-96e7-56b99df116b7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xvgn2" May 27 21:38:30.010735 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:30.010710 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5grv\" (UniqueName: \"kubernetes.io/projected/f1233a43-b5ac-45a9-96e7-56b99df116b7-kube-api-access-g5grv\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xvgn2\" (UID: \"f1233a43-b5ac-45a9-96e7-56b99df116b7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xvgn2" May 27 21:38:30.163105 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:30.163025 2564 generic.go:358] "Generic (PLEG): container finished" podID="8515de3f-c993-4bd8-bb41-91ec6d3bbf2b" containerID="bb52372c1e67f5914b24ef866d5285f34e22e1c4f0c0c7d4a77907f98aebed5f" exitCode=0 May 27 21:38:30.163251 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:30.163118 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tqm2b" event={"ID":"8515de3f-c993-4bd8-bb41-91ec6d3bbf2b","Type":"ContainerDied","Data":"bb52372c1e67f5914b24ef866d5285f34e22e1c4f0c0c7d4a77907f98aebed5f"} May 27 21:38:30.163251 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:30.163155 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tqm2b" event={"ID":"8515de3f-c993-4bd8-bb41-91ec6d3bbf2b","Type":"ContainerStarted","Data":"f59c0b725dd21d870e2e445294faac0c9b42a5110707110d53d9bbdb8ea0c400"} May 27 21:38:30.191582 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:30.191556 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xvgn2" May 27 21:38:30.273915 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:30.273874 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75975psw"] May 27 21:38:30.278717 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:30.278695 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75975psw" May 27 21:38:30.286699 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:30.286654 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75975psw"] May 27 21:38:30.304396 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:30.304367 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/844b76fb-f722-4de9-a98d-fb539fead6f0-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75975psw\" (UID: \"844b76fb-f722-4de9-a98d-fb539fead6f0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75975psw" May 27 21:38:30.304565 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:30.304420 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgqtj\" (UniqueName: \"kubernetes.io/projected/844b76fb-f722-4de9-a98d-fb539fead6f0-kube-api-access-bgqtj\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75975psw\" (UID: \"844b76fb-f722-4de9-a98d-fb539fead6f0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75975psw" May 27 21:38:30.304565 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:30.304506 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/844b76fb-f722-4de9-a98d-fb539fead6f0-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75975psw\" (UID: \"844b76fb-f722-4de9-a98d-fb539fead6f0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75975psw" May 27 21:38:30.322163 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:30.322137 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xvgn2"] May 27 21:38:30.323580 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:38:30.323545 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1233a43_b5ac_45a9_96e7_56b99df116b7.slice/crio-37aa6bb5faaf6f1bee44a99df440d70614277a07932198c4f38783b9595db99d WatchSource:0}: Error finding container 37aa6bb5faaf6f1bee44a99df440d70614277a07932198c4f38783b9595db99d: Status 404 returned error can't find the container with id 37aa6bb5faaf6f1bee44a99df440d70614277a07932198c4f38783b9595db99d May 27 21:38:30.405246 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:30.405210 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bgqtj\" (UniqueName: \"kubernetes.io/projected/844b76fb-f722-4de9-a98d-fb539fead6f0-kube-api-access-bgqtj\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75975psw\" (UID: \"844b76fb-f722-4de9-a98d-fb539fead6f0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75975psw" May 27 21:38:30.405417 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:30.405259 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/844b76fb-f722-4de9-a98d-fb539fead6f0-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75975psw\" (UID: \"844b76fb-f722-4de9-a98d-fb539fead6f0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75975psw" May 27 21:38:30.405417 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:30.405381 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/844b76fb-f722-4de9-a98d-fb539fead6f0-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75975psw\" (UID: \"844b76fb-f722-4de9-a98d-fb539fead6f0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75975psw" May 27 21:38:30.405661 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:30.405641 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/844b76fb-f722-4de9-a98d-fb539fead6f0-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75975psw\" (UID: \"844b76fb-f722-4de9-a98d-fb539fead6f0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75975psw" May 27 21:38:30.405707 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:30.405690 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/844b76fb-f722-4de9-a98d-fb539fead6f0-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75975psw\" (UID: \"844b76fb-f722-4de9-a98d-fb539fead6f0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75975psw" May 27 21:38:30.415114 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:30.415056 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgqtj\" (UniqueName: \"kubernetes.io/projected/844b76fb-f722-4de9-a98d-fb539fead6f0-kube-api-access-bgqtj\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75975psw\" (UID: \"844b76fb-f722-4de9-a98d-fb539fead6f0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75975psw" May 27 21:38:30.591173 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:30.591135 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75975psw" May 27 21:38:30.682018 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:30.681985 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dv7fr"] May 27 21:38:30.687317 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:30.687294 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dv7fr" May 27 21:38:30.692863 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:30.692793 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dv7fr"] May 27 21:38:30.711776 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:30.710323 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bad5a47a-1b3c-4c5b-88a3-b5972403452f-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dv7fr\" (UID: \"bad5a47a-1b3c-4c5b-88a3-b5972403452f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dv7fr" May 27 21:38:30.711776 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:30.710378 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bad5a47a-1b3c-4c5b-88a3-b5972403452f-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dv7fr\" (UID: \"bad5a47a-1b3c-4c5b-88a3-b5972403452f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dv7fr" May 27 21:38:30.711776 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:30.710443 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mczt\" (UniqueName: \"kubernetes.io/projected/bad5a47a-1b3c-4c5b-88a3-b5972403452f-kube-api-access-9mczt\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dv7fr\" (UID: \"bad5a47a-1b3c-4c5b-88a3-b5972403452f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dv7fr" May 27 21:38:30.732710 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:30.732684 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75975psw"] May 27 21:38:30.734755 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:38:30.734732 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod844b76fb_f722_4de9_a98d_fb539fead6f0.slice/crio-c0173def2435d004e3b4d033f86ac9abdcf525f01465529fc4035e7012ee65c2 WatchSource:0}: Error finding container c0173def2435d004e3b4d033f86ac9abdcf525f01465529fc4035e7012ee65c2: Status 404 returned error can't find the container with id c0173def2435d004e3b4d033f86ac9abdcf525f01465529fc4035e7012ee65c2 May 27 21:38:30.811639 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:30.811603 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bad5a47a-1b3c-4c5b-88a3-b5972403452f-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dv7fr\" (UID: \"bad5a47a-1b3c-4c5b-88a3-b5972403452f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dv7fr" May 27 21:38:30.811639 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:30.811645 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bad5a47a-1b3c-4c5b-88a3-b5972403452f-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dv7fr\" (UID: \"bad5a47a-1b3c-4c5b-88a3-b5972403452f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dv7fr" May 27 21:38:30.811864 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:30.811676 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9mczt\" (UniqueName: \"kubernetes.io/projected/bad5a47a-1b3c-4c5b-88a3-b5972403452f-kube-api-access-9mczt\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dv7fr\" (UID: \"bad5a47a-1b3c-4c5b-88a3-b5972403452f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dv7fr" May 27 21:38:30.812027 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:30.812003 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bad5a47a-1b3c-4c5b-88a3-b5972403452f-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dv7fr\" (UID: \"bad5a47a-1b3c-4c5b-88a3-b5972403452f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dv7fr" May 27 21:38:30.812096 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:30.812045 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bad5a47a-1b3c-4c5b-88a3-b5972403452f-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dv7fr\" (UID: \"bad5a47a-1b3c-4c5b-88a3-b5972403452f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dv7fr" May 27 21:38:30.821182 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:30.821158 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mczt\" (UniqueName: \"kubernetes.io/projected/bad5a47a-1b3c-4c5b-88a3-b5972403452f-kube-api-access-9mczt\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dv7fr\" (UID: \"bad5a47a-1b3c-4c5b-88a3-b5972403452f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dv7fr" May 27 21:38:31.001900 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:31.001831 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dv7fr" May 27 21:38:31.149855 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:31.149828 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dv7fr"] May 27 21:38:31.151622 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:38:31.151594 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbad5a47a_1b3c_4c5b_88a3_b5972403452f.slice/crio-db640762378017cd1935963e4454b1eab41bf58d58c997075ea252df2f3cb453 WatchSource:0}: Error finding container db640762378017cd1935963e4454b1eab41bf58d58c997075ea252df2f3cb453: Status 404 returned error can't find the container with id db640762378017cd1935963e4454b1eab41bf58d58c997075ea252df2f3cb453 May 27 21:38:31.168177 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:31.168154 2564 generic.go:358] "Generic (PLEG): container finished" podID="f1233a43-b5ac-45a9-96e7-56b99df116b7" containerID="fb2046ca839c77767da1f82d41083b89e27d93eea9eb009375baa51f6dfc71a4" exitCode=0 May 27 21:38:31.168302 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:31.168230 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xvgn2" event={"ID":"f1233a43-b5ac-45a9-96e7-56b99df116b7","Type":"ContainerDied","Data":"fb2046ca839c77767da1f82d41083b89e27d93eea9eb009375baa51f6dfc71a4"} May 27 21:38:31.168302 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:31.168294 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xvgn2" event={"ID":"f1233a43-b5ac-45a9-96e7-56b99df116b7","Type":"ContainerStarted","Data":"37aa6bb5faaf6f1bee44a99df440d70614277a07932198c4f38783b9595db99d"} May 27 21:38:31.170102 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:31.170077 2564 generic.go:358] "Generic (PLEG): container finished" podID="8515de3f-c993-4bd8-bb41-91ec6d3bbf2b" containerID="07185fc750d253083d1627b07c3219477c7d30d4d2d3af559e504dbd79c96ddb" exitCode=0 May 27 21:38:31.170192 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:31.170162 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tqm2b" event={"ID":"8515de3f-c993-4bd8-bb41-91ec6d3bbf2b","Type":"ContainerDied","Data":"07185fc750d253083d1627b07c3219477c7d30d4d2d3af559e504dbd79c96ddb"} May 27 21:38:31.171578 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:31.171535 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dv7fr" event={"ID":"bad5a47a-1b3c-4c5b-88a3-b5972403452f","Type":"ContainerStarted","Data":"db640762378017cd1935963e4454b1eab41bf58d58c997075ea252df2f3cb453"} May 27 21:38:31.172979 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:31.172950 2564 generic.go:358] "Generic (PLEG): container finished" podID="844b76fb-f722-4de9-a98d-fb539fead6f0" containerID="1c4db195414d0cd1f91baad811ee60a7b53159ab7f1196816e2114ac1209f820" exitCode=0 May 27 21:38:31.173087 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:31.173068 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75975psw" event={"ID":"844b76fb-f722-4de9-a98d-fb539fead6f0","Type":"ContainerDied","Data":"1c4db195414d0cd1f91baad811ee60a7b53159ab7f1196816e2114ac1209f820"} May 27 21:38:31.173159 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:31.173090 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75975psw" event={"ID":"844b76fb-f722-4de9-a98d-fb539fead6f0","Type":"ContainerStarted","Data":"c0173def2435d004e3b4d033f86ac9abdcf525f01465529fc4035e7012ee65c2"} May 27 21:38:32.181193 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:32.181158 2564 generic.go:358] "Generic (PLEG): container finished" podID="f1233a43-b5ac-45a9-96e7-56b99df116b7" containerID="1e5493795782947af6bed000bb04d14e72ff848ee8d10804d3cac95e579902fa" exitCode=0 May 27 21:38:32.181615 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:32.181243 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xvgn2" event={"ID":"f1233a43-b5ac-45a9-96e7-56b99df116b7","Type":"ContainerDied","Data":"1e5493795782947af6bed000bb04d14e72ff848ee8d10804d3cac95e579902fa"} May 27 21:38:32.183314 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:32.183291 2564 generic.go:358] "Generic (PLEG): container finished" podID="8515de3f-c993-4bd8-bb41-91ec6d3bbf2b" containerID="5ec05995e5a2d1535c14f2e1edca7bfd653e9c274bff137d5eb3e78bbc3eb77c" exitCode=0 May 27 21:38:32.183430 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:32.183373 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tqm2b" event={"ID":"8515de3f-c993-4bd8-bb41-91ec6d3bbf2b","Type":"ContainerDied","Data":"5ec05995e5a2d1535c14f2e1edca7bfd653e9c274bff137d5eb3e78bbc3eb77c"} May 27 21:38:32.184616 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:32.184597 2564 generic.go:358] "Generic (PLEG): container finished" podID="bad5a47a-1b3c-4c5b-88a3-b5972403452f" containerID="b7df14e838e5f8abd40fee9940b39eb30d314bc2540498e39addb326420accb3" exitCode=0 May 27 21:38:32.184720 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:32.184647 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dv7fr" event={"ID":"bad5a47a-1b3c-4c5b-88a3-b5972403452f","Type":"ContainerDied","Data":"b7df14e838e5f8abd40fee9940b39eb30d314bc2540498e39addb326420accb3"} May 27 21:38:32.186626 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:32.186505 2564 generic.go:358] "Generic (PLEG): container finished" podID="844b76fb-f722-4de9-a98d-fb539fead6f0" containerID="1c544dfcb0b4ffa37a0b38f8bf977cc74f8f698074846e6810e57fe0b54cd7c6" exitCode=0 May 27 21:38:32.186626 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:32.186578 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75975psw" event={"ID":"844b76fb-f722-4de9-a98d-fb539fead6f0","Type":"ContainerDied","Data":"1c544dfcb0b4ffa37a0b38f8bf977cc74f8f698074846e6810e57fe0b54cd7c6"} May 27 21:38:33.191910 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:33.191869 2564 generic.go:358] "Generic (PLEG): container finished" podID="f1233a43-b5ac-45a9-96e7-56b99df116b7" containerID="db1fa13f928ae4e15db7c52323e36de3154a5f43569d8c84ca8d0624afd5dc28" exitCode=0 May 27 21:38:33.192324 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:33.191944 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xvgn2" event={"ID":"f1233a43-b5ac-45a9-96e7-56b99df116b7","Type":"ContainerDied","Data":"db1fa13f928ae4e15db7c52323e36de3154a5f43569d8c84ca8d0624afd5dc28"} May 27 21:38:33.193659 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:33.193629 2564 generic.go:358] "Generic (PLEG): container finished" podID="bad5a47a-1b3c-4c5b-88a3-b5972403452f" containerID="feece5a4cb82eb2e167bf317b6c167ea6c9f7ead4ecc1148b9e089cfd0f57a91" exitCode=0 May 27 21:38:33.193659 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:33.193654 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dv7fr" event={"ID":"bad5a47a-1b3c-4c5b-88a3-b5972403452f","Type":"ContainerDied","Data":"feece5a4cb82eb2e167bf317b6c167ea6c9f7ead4ecc1148b9e089cfd0f57a91"} May 27 21:38:33.195746 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:33.195724 2564 generic.go:358] "Generic (PLEG): container finished" podID="844b76fb-f722-4de9-a98d-fb539fead6f0" containerID="89d557c06a55567e8f3ad5d253617eb9de09380ce9168f48c7bb20231fce43db" exitCode=0 May 27 21:38:33.195849 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:33.195808 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75975psw" event={"ID":"844b76fb-f722-4de9-a98d-fb539fead6f0","Type":"ContainerDied","Data":"89d557c06a55567e8f3ad5d253617eb9de09380ce9168f48c7bb20231fce43db"} May 27 21:38:33.327782 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:33.327758 2564 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tqm2b" May 27 21:38:33.436679 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:33.436652 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8515de3f-c993-4bd8-bb41-91ec6d3bbf2b-util\") pod \"8515de3f-c993-4bd8-bb41-91ec6d3bbf2b\" (UID: \"8515de3f-c993-4bd8-bb41-91ec6d3bbf2b\") " May 27 21:38:33.436794 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:33.436707 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8515de3f-c993-4bd8-bb41-91ec6d3bbf2b-bundle\") pod \"8515de3f-c993-4bd8-bb41-91ec6d3bbf2b\" (UID: \"8515de3f-c993-4bd8-bb41-91ec6d3bbf2b\") " May 27 21:38:33.436794 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:33.436728 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tnd7t\" (UniqueName: \"kubernetes.io/projected/8515de3f-c993-4bd8-bb41-91ec6d3bbf2b-kube-api-access-tnd7t\") pod \"8515de3f-c993-4bd8-bb41-91ec6d3bbf2b\" (UID: \"8515de3f-c993-4bd8-bb41-91ec6d3bbf2b\") " May 27 21:38:33.437217 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:33.437189 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8515de3f-c993-4bd8-bb41-91ec6d3bbf2b-bundle" (OuterVolumeSpecName: "bundle") pod "8515de3f-c993-4bd8-bb41-91ec6d3bbf2b" (UID: "8515de3f-c993-4bd8-bb41-91ec6d3bbf2b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 27 21:38:33.438884 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:33.438862 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8515de3f-c993-4bd8-bb41-91ec6d3bbf2b-kube-api-access-tnd7t" (OuterVolumeSpecName: "kube-api-access-tnd7t") pod "8515de3f-c993-4bd8-bb41-91ec6d3bbf2b" (UID: "8515de3f-c993-4bd8-bb41-91ec6d3bbf2b"). InnerVolumeSpecName "kube-api-access-tnd7t". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 27 21:38:33.442341 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:33.442319 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8515de3f-c993-4bd8-bb41-91ec6d3bbf2b-util" (OuterVolumeSpecName: "util") pod "8515de3f-c993-4bd8-bb41-91ec6d3bbf2b" (UID: "8515de3f-c993-4bd8-bb41-91ec6d3bbf2b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 27 21:38:33.537491 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:33.537431 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8515de3f-c993-4bd8-bb41-91ec6d3bbf2b-util\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:38:33.537491 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:33.537456 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8515de3f-c993-4bd8-bb41-91ec6d3bbf2b-bundle\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:38:33.537491 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:33.537465 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tnd7t\" (UniqueName: \"kubernetes.io/projected/8515de3f-c993-4bd8-bb41-91ec6d3bbf2b-kube-api-access-tnd7t\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:38:34.204379 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:34.204339 2564 generic.go:358] "Generic (PLEG): container finished" podID="bad5a47a-1b3c-4c5b-88a3-b5972403452f" containerID="770c9a8675444c415224b226348c84647e0f3716a2988d57d32ef4216506083d" exitCode=0 May 27 21:38:34.204877 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:34.204420 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dv7fr" event={"ID":"bad5a47a-1b3c-4c5b-88a3-b5972403452f","Type":"ContainerDied","Data":"770c9a8675444c415224b226348c84647e0f3716a2988d57d32ef4216506083d"} May 27 21:38:34.206077 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:34.206050 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tqm2b" event={"ID":"8515de3f-c993-4bd8-bb41-91ec6d3bbf2b","Type":"ContainerDied","Data":"f59c0b725dd21d870e2e445294faac0c9b42a5110707110d53d9bbdb8ea0c400"} May 27 21:38:34.206173 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:34.206082 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f59c0b725dd21d870e2e445294faac0c9b42a5110707110d53d9bbdb8ea0c400" May 27 21:38:34.206319 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:34.206303 2564 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tqm2b" May 27 21:38:34.371219 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:34.371200 2564 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75975psw" May 27 21:38:34.374483 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:34.374464 2564 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xvgn2" May 27 21:38:34.445463 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:34.445429 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g5grv\" (UniqueName: \"kubernetes.io/projected/f1233a43-b5ac-45a9-96e7-56b99df116b7-kube-api-access-g5grv\") pod \"f1233a43-b5ac-45a9-96e7-56b99df116b7\" (UID: \"f1233a43-b5ac-45a9-96e7-56b99df116b7\") " May 27 21:38:34.445626 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:34.445490 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f1233a43-b5ac-45a9-96e7-56b99df116b7-util\") pod \"f1233a43-b5ac-45a9-96e7-56b99df116b7\" (UID: \"f1233a43-b5ac-45a9-96e7-56b99df116b7\") " May 27 21:38:34.445626 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:34.445539 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f1233a43-b5ac-45a9-96e7-56b99df116b7-bundle\") pod \"f1233a43-b5ac-45a9-96e7-56b99df116b7\" (UID: \"f1233a43-b5ac-45a9-96e7-56b99df116b7\") " May 27 21:38:34.445626 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:34.445572 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/844b76fb-f722-4de9-a98d-fb539fead6f0-util\") pod \"844b76fb-f722-4de9-a98d-fb539fead6f0\" (UID: \"844b76fb-f722-4de9-a98d-fb539fead6f0\") " May 27 21:38:34.445626 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:34.445609 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/844b76fb-f722-4de9-a98d-fb539fead6f0-bundle\") pod \"844b76fb-f722-4de9-a98d-fb539fead6f0\" (UID: \"844b76fb-f722-4de9-a98d-fb539fead6f0\") " May 27 21:38:34.445861 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:34.445683 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bgqtj\" (UniqueName: \"kubernetes.io/projected/844b76fb-f722-4de9-a98d-fb539fead6f0-kube-api-access-bgqtj\") pod \"844b76fb-f722-4de9-a98d-fb539fead6f0\" (UID: \"844b76fb-f722-4de9-a98d-fb539fead6f0\") " May 27 21:38:34.446084 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:34.446056 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1233a43-b5ac-45a9-96e7-56b99df116b7-bundle" (OuterVolumeSpecName: "bundle") pod "f1233a43-b5ac-45a9-96e7-56b99df116b7" (UID: "f1233a43-b5ac-45a9-96e7-56b99df116b7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 27 21:38:34.446147 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:34.446131 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/844b76fb-f722-4de9-a98d-fb539fead6f0-bundle" (OuterVolumeSpecName: "bundle") pod "844b76fb-f722-4de9-a98d-fb539fead6f0" (UID: "844b76fb-f722-4de9-a98d-fb539fead6f0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 27 21:38:34.447785 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:34.447754 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1233a43-b5ac-45a9-96e7-56b99df116b7-kube-api-access-g5grv" (OuterVolumeSpecName: "kube-api-access-g5grv") pod "f1233a43-b5ac-45a9-96e7-56b99df116b7" (UID: "f1233a43-b5ac-45a9-96e7-56b99df116b7"). InnerVolumeSpecName "kube-api-access-g5grv". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 27 21:38:34.448015 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:34.447990 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/844b76fb-f722-4de9-a98d-fb539fead6f0-kube-api-access-bgqtj" (OuterVolumeSpecName: "kube-api-access-bgqtj") pod "844b76fb-f722-4de9-a98d-fb539fead6f0" (UID: "844b76fb-f722-4de9-a98d-fb539fead6f0"). InnerVolumeSpecName "kube-api-access-bgqtj". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 27 21:38:34.451699 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:34.451671 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1233a43-b5ac-45a9-96e7-56b99df116b7-util" (OuterVolumeSpecName: "util") pod "f1233a43-b5ac-45a9-96e7-56b99df116b7" (UID: "f1233a43-b5ac-45a9-96e7-56b99df116b7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 27 21:38:34.453360 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:34.453331 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/844b76fb-f722-4de9-a98d-fb539fead6f0-util" (OuterVolumeSpecName: "util") pod "844b76fb-f722-4de9-a98d-fb539fead6f0" (UID: "844b76fb-f722-4de9-a98d-fb539fead6f0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 27 21:38:34.547183 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:34.547139 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f1233a43-b5ac-45a9-96e7-56b99df116b7-bundle\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:38:34.547183 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:34.547177 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/844b76fb-f722-4de9-a98d-fb539fead6f0-util\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:38:34.547183 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:34.547187 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/844b76fb-f722-4de9-a98d-fb539fead6f0-bundle\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:38:34.547460 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:34.547196 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bgqtj\" (UniqueName: \"kubernetes.io/projected/844b76fb-f722-4de9-a98d-fb539fead6f0-kube-api-access-bgqtj\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:38:34.547460 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:34.547207 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g5grv\" (UniqueName: \"kubernetes.io/projected/f1233a43-b5ac-45a9-96e7-56b99df116b7-kube-api-access-g5grv\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:38:34.547460 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:34.547215 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f1233a43-b5ac-45a9-96e7-56b99df116b7-util\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:38:35.211155 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:35.211127 2564 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75975psw" May 27 21:38:35.211636 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:35.211150 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75975psw" event={"ID":"844b76fb-f722-4de9-a98d-fb539fead6f0","Type":"ContainerDied","Data":"c0173def2435d004e3b4d033f86ac9abdcf525f01465529fc4035e7012ee65c2"} May 27 21:38:35.211636 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:35.211187 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c0173def2435d004e3b4d033f86ac9abdcf525f01465529fc4035e7012ee65c2" May 27 21:38:35.212997 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:35.212973 2564 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xvgn2" May 27 21:38:35.213136 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:35.213006 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xvgn2" event={"ID":"f1233a43-b5ac-45a9-96e7-56b99df116b7","Type":"ContainerDied","Data":"37aa6bb5faaf6f1bee44a99df440d70614277a07932198c4f38783b9595db99d"} May 27 21:38:35.213136 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:35.213029 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="37aa6bb5faaf6f1bee44a99df440d70614277a07932198c4f38783b9595db99d" May 27 21:38:35.336307 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:35.336286 2564 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dv7fr" May 27 21:38:35.455238 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:35.455208 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9mczt\" (UniqueName: \"kubernetes.io/projected/bad5a47a-1b3c-4c5b-88a3-b5972403452f-kube-api-access-9mczt\") pod \"bad5a47a-1b3c-4c5b-88a3-b5972403452f\" (UID: \"bad5a47a-1b3c-4c5b-88a3-b5972403452f\") " May 27 21:38:35.455448 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:35.455257 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bad5a47a-1b3c-4c5b-88a3-b5972403452f-bundle\") pod \"bad5a47a-1b3c-4c5b-88a3-b5972403452f\" (UID: \"bad5a47a-1b3c-4c5b-88a3-b5972403452f\") " May 27 21:38:35.455448 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:35.455370 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bad5a47a-1b3c-4c5b-88a3-b5972403452f-util\") pod \"bad5a47a-1b3c-4c5b-88a3-b5972403452f\" (UID: \"bad5a47a-1b3c-4c5b-88a3-b5972403452f\") " May 27 21:38:35.455941 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:35.455907 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bad5a47a-1b3c-4c5b-88a3-b5972403452f-bundle" (OuterVolumeSpecName: "bundle") pod "bad5a47a-1b3c-4c5b-88a3-b5972403452f" (UID: "bad5a47a-1b3c-4c5b-88a3-b5972403452f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 27 21:38:35.457302 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:35.457279 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bad5a47a-1b3c-4c5b-88a3-b5972403452f-kube-api-access-9mczt" (OuterVolumeSpecName: "kube-api-access-9mczt") pod "bad5a47a-1b3c-4c5b-88a3-b5972403452f" (UID: "bad5a47a-1b3c-4c5b-88a3-b5972403452f"). InnerVolumeSpecName "kube-api-access-9mczt". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 27 21:38:35.460721 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:35.460695 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bad5a47a-1b3c-4c5b-88a3-b5972403452f-util" (OuterVolumeSpecName: "util") pod "bad5a47a-1b3c-4c5b-88a3-b5972403452f" (UID: "bad5a47a-1b3c-4c5b-88a3-b5972403452f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 27 21:38:35.556648 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:35.556620 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bad5a47a-1b3c-4c5b-88a3-b5972403452f-bundle\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:38:35.556648 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:35.556643 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bad5a47a-1b3c-4c5b-88a3-b5972403452f-util\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:38:35.556648 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:35.556654 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9mczt\" (UniqueName: \"kubernetes.io/projected/bad5a47a-1b3c-4c5b-88a3-b5972403452f-kube-api-access-9mczt\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:38:36.218348 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:36.218308 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dv7fr" event={"ID":"bad5a47a-1b3c-4c5b-88a3-b5972403452f","Type":"ContainerDied","Data":"db640762378017cd1935963e4454b1eab41bf58d58c997075ea252df2f3cb453"} May 27 21:38:36.218348 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:36.218351 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="db640762378017cd1935963e4454b1eab41bf58d58c997075ea252df2f3cb453" May 27 21:38:36.218762 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:36.218324 2564 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dv7fr" May 27 21:38:47.013421 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:47.013389 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-d9gcd_16b8cf1e-1181-43fc-92e0-635b8523248d/console-operator/2.log" May 27 21:38:47.016258 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:47.016225 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-d9gcd_16b8cf1e-1181-43fc-92e0-635b8523248d/console-operator/2.log" May 27 21:38:49.184971 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:49.184932 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-657f44b778-lznn9"] May 27 21:38:49.185389 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:49.185285 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8515de3f-c993-4bd8-bb41-91ec6d3bbf2b" containerName="extract" May 27 21:38:49.185389 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:49.185301 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8515de3f-c993-4bd8-bb41-91ec6d3bbf2b" containerName="extract" May 27 21:38:49.185389 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:49.185314 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="844b76fb-f722-4de9-a98d-fb539fead6f0" containerName="extract" May 27 21:38:49.185389 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:49.185322 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="844b76fb-f722-4de9-a98d-fb539fead6f0" containerName="extract" May 27 21:38:49.185389 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:49.185330 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bad5a47a-1b3c-4c5b-88a3-b5972403452f" containerName="util" May 27 21:38:49.185389 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:49.185335 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="bad5a47a-1b3c-4c5b-88a3-b5972403452f" containerName="util" May 27 21:38:49.185389 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:49.185344 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8515de3f-c993-4bd8-bb41-91ec6d3bbf2b" containerName="util" May 27 21:38:49.185389 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:49.185349 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8515de3f-c993-4bd8-bb41-91ec6d3bbf2b" containerName="util" May 27 21:38:49.185389 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:49.185358 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8515de3f-c993-4bd8-bb41-91ec6d3bbf2b" containerName="pull" May 27 21:38:49.185389 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:49.185363 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8515de3f-c993-4bd8-bb41-91ec6d3bbf2b" containerName="pull" May 27 21:38:49.185389 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:49.185370 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1233a43-b5ac-45a9-96e7-56b99df116b7" containerName="extract" May 27 21:38:49.185389 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:49.185375 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1233a43-b5ac-45a9-96e7-56b99df116b7" containerName="extract" May 27 21:38:49.185389 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:49.185383 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bad5a47a-1b3c-4c5b-88a3-b5972403452f" containerName="extract" May 27 21:38:49.185389 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:49.185388 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="bad5a47a-1b3c-4c5b-88a3-b5972403452f" containerName="extract" May 27 21:38:49.185389 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:49.185394 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1233a43-b5ac-45a9-96e7-56b99df116b7" containerName="util" May 27 21:38:49.185389 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:49.185399 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1233a43-b5ac-45a9-96e7-56b99df116b7" containerName="util" May 27 21:38:49.185844 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:49.185405 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1233a43-b5ac-45a9-96e7-56b99df116b7" containerName="pull" May 27 21:38:49.185844 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:49.185410 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1233a43-b5ac-45a9-96e7-56b99df116b7" containerName="pull" May 27 21:38:49.185844 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:49.185415 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bad5a47a-1b3c-4c5b-88a3-b5972403452f" containerName="pull" May 27 21:38:49.185844 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:49.185419 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="bad5a47a-1b3c-4c5b-88a3-b5972403452f" containerName="pull" May 27 21:38:49.185844 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:49.185432 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="844b76fb-f722-4de9-a98d-fb539fead6f0" containerName="pull" May 27 21:38:49.185844 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:49.185437 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="844b76fb-f722-4de9-a98d-fb539fead6f0" containerName="pull" May 27 21:38:49.185844 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:49.185443 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="844b76fb-f722-4de9-a98d-fb539fead6f0" containerName="util" May 27 21:38:49.185844 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:49.185448 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="844b76fb-f722-4de9-a98d-fb539fead6f0" containerName="util" May 27 21:38:49.185844 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:49.185522 2564 memory_manager.go:356] "RemoveStaleState removing state" podUID="8515de3f-c993-4bd8-bb41-91ec6d3bbf2b" containerName="extract" May 27 21:38:49.185844 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:49.185532 2564 memory_manager.go:356] "RemoveStaleState removing state" podUID="f1233a43-b5ac-45a9-96e7-56b99df116b7" containerName="extract" May 27 21:38:49.185844 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:49.185539 2564 memory_manager.go:356] "RemoveStaleState removing state" podUID="bad5a47a-1b3c-4c5b-88a3-b5972403452f" containerName="extract" May 27 21:38:49.185844 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:49.185546 2564 memory_manager.go:356] "RemoveStaleState removing state" podUID="844b76fb-f722-4de9-a98d-fb539fead6f0" containerName="extract" May 27 21:38:49.189750 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:49.189727 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-657f44b778-lznn9" May 27 21:38:49.192705 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:49.192686 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-8sfbh\"" May 27 21:38:49.193855 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:49.193833 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" May 27 21:38:49.193960 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:49.193885 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" May 27 21:38:49.201813 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:49.201791 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-657f44b778-lznn9"] May 27 21:38:49.273705 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:49.273678 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfnj8\" (UniqueName: \"kubernetes.io/projected/b3cc5b72-1ec8-4ebe-b7bf-9c9ae0be8e3a-kube-api-access-pfnj8\") pod \"authorino-operator-657f44b778-lznn9\" (UID: \"b3cc5b72-1ec8-4ebe-b7bf-9c9ae0be8e3a\") " pod="kuadrant-system/authorino-operator-657f44b778-lznn9" May 27 21:38:49.375208 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:49.375181 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pfnj8\" (UniqueName: \"kubernetes.io/projected/b3cc5b72-1ec8-4ebe-b7bf-9c9ae0be8e3a-kube-api-access-pfnj8\") pod \"authorino-operator-657f44b778-lznn9\" (UID: \"b3cc5b72-1ec8-4ebe-b7bf-9c9ae0be8e3a\") " pod="kuadrant-system/authorino-operator-657f44b778-lznn9" May 27 21:38:49.386592 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:49.386566 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfnj8\" (UniqueName: \"kubernetes.io/projected/b3cc5b72-1ec8-4ebe-b7bf-9c9ae0be8e3a-kube-api-access-pfnj8\") pod \"authorino-operator-657f44b778-lznn9\" (UID: \"b3cc5b72-1ec8-4ebe-b7bf-9c9ae0be8e3a\") " pod="kuadrant-system/authorino-operator-657f44b778-lznn9" May 27 21:38:49.501129 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:49.501057 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-657f44b778-lznn9" May 27 21:38:49.632073 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:49.632047 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-657f44b778-lznn9"] May 27 21:38:49.633279 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:38:49.633232 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3cc5b72_1ec8_4ebe_b7bf_9c9ae0be8e3a.slice/crio-ce7f63d73b512884ef408aab52a6603fd32a95a05638992707ee50a77d176c0e WatchSource:0}: Error finding container ce7f63d73b512884ef408aab52a6603fd32a95a05638992707ee50a77d176c0e: Status 404 returned error can't find the container with id ce7f63d73b512884ef408aab52a6603fd32a95a05638992707ee50a77d176c0e May 27 21:38:50.271186 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:50.271145 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-657f44b778-lznn9" event={"ID":"b3cc5b72-1ec8-4ebe-b7bf-9c9ae0be8e3a","Type":"ContainerStarted","Data":"ce7f63d73b512884ef408aab52a6603fd32a95a05638992707ee50a77d176c0e"} May 27 21:38:52.283339 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:52.283302 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-657f44b778-lznn9" event={"ID":"b3cc5b72-1ec8-4ebe-b7bf-9c9ae0be8e3a","Type":"ContainerStarted","Data":"dbae64d516c3ef0ed0177ea934229caf8d1ec1467fe34fb0a5903dcb73423905"} May 27 21:38:52.283712 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:52.283380 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-657f44b778-lznn9" May 27 21:38:52.301654 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:38:52.301605 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-657f44b778-lznn9" podStartSLOduration=1.237687451 podStartE2EDuration="3.301591851s" podCreationTimestamp="2026-05-27 21:38:49 +0000 UTC" firstStartedPulling="2026-05-27 21:38:49.636182328 +0000 UTC m=+603.110387039" lastFinishedPulling="2026-05-27 21:38:51.700086731 +0000 UTC m=+605.174291439" observedRunningTime="2026-05-27 21:38:52.299851725 +0000 UTC m=+605.774056455" watchObservedRunningTime="2026-05-27 21:38:52.301591851 +0000 UTC m=+605.775796558" May 27 21:39:03.288949 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:39:03.288909 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-657f44b778-lznn9" May 27 21:39:15.938231 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:39:15.938195 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-pqxpq"] May 27 21:39:15.941515 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:39:15.941499 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-pqxpq" May 27 21:39:15.944773 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:39:15.944752 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-6wcbq\"" May 27 21:39:15.957978 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:39:15.957955 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-pqxpq"] May 27 21:39:16.007024 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:39:16.006997 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tp78t\" (UniqueName: \"kubernetes.io/projected/71c6995f-03ab-442a-8b81-846260fbd05e-kube-api-access-tp78t\") pod \"kuadrant-operator-controller-manager-84b657d985-pqxpq\" (UID: \"71c6995f-03ab-442a-8b81-846260fbd05e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-pqxpq" May 27 21:39:16.007149 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:39:16.007125 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/71c6995f-03ab-442a-8b81-846260fbd05e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-84b657d985-pqxpq\" (UID: \"71c6995f-03ab-442a-8b81-846260fbd05e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-pqxpq" May 27 21:39:16.074958 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:39:16.074924 2564 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-pqxpq"] May 27 21:39:16.075170 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:39:16.075150 2564 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-tp78t], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-pqxpq" podUID="71c6995f-03ab-442a-8b81-846260fbd05e" May 27 21:39:16.108383 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:39:16.108351 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/71c6995f-03ab-442a-8b81-846260fbd05e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-84b657d985-pqxpq\" (UID: \"71c6995f-03ab-442a-8b81-846260fbd05e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-pqxpq" May 27 21:39:16.108519 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:39:16.108406 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tp78t\" (UniqueName: \"kubernetes.io/projected/71c6995f-03ab-442a-8b81-846260fbd05e-kube-api-access-tp78t\") pod \"kuadrant-operator-controller-manager-84b657d985-pqxpq\" (UID: \"71c6995f-03ab-442a-8b81-846260fbd05e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-pqxpq" May 27 21:39:16.108797 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:39:16.108771 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/71c6995f-03ab-442a-8b81-846260fbd05e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-84b657d985-pqxpq\" (UID: \"71c6995f-03ab-442a-8b81-846260fbd05e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-pqxpq" May 27 21:39:16.129204 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:39:16.129180 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tp78t\" (UniqueName: \"kubernetes.io/projected/71c6995f-03ab-442a-8b81-846260fbd05e-kube-api-access-tp78t\") pod \"kuadrant-operator-controller-manager-84b657d985-pqxpq\" (UID: \"71c6995f-03ab-442a-8b81-846260fbd05e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-pqxpq" May 27 21:39:16.218998 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:39:16.218925 2564 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-pqxpq"] May 27 21:39:16.221290 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:39:16.221248 2564 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-pqxpq"] May 27 21:39:16.288366 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:39:16.288338 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-pt7th"] May 27 21:39:16.291690 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:39:16.291673 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-pt7th" May 27 21:39:16.295964 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:39:16.295938 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-hkqtz\"" May 27 21:39:16.311350 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:39:16.311323 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-pt7th"] May 27 21:39:16.381649 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:39:16.377581 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-pqxpq" May 27 21:39:16.382124 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:39:16.382094 2564 status_manager.go:895] "Failed to get status for pod" podUID="71c6995f-03ab-442a-8b81-846260fbd05e" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-pqxpq" err="pods \"kuadrant-operator-controller-manager-84b657d985-pqxpq\" is forbidden: User \"system:node:ip-10-0-130-246.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-246.ec2.internal' and this object" May 27 21:39:16.385987 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:39:16.385968 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-pqxpq" May 27 21:39:16.388186 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:39:16.388162 2564 status_manager.go:895] "Failed to get status for pod" podUID="71c6995f-03ab-442a-8b81-846260fbd05e" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-pqxpq" err="pods \"kuadrant-operator-controller-manager-84b657d985-pqxpq\" is forbidden: User \"system:node:ip-10-0-130-246.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-246.ec2.internal' and this object" May 27 21:39:16.411021 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:39:16.410998 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/71c6995f-03ab-442a-8b81-846260fbd05e-extensions-socket-volume\") pod \"71c6995f-03ab-442a-8b81-846260fbd05e\" (UID: \"71c6995f-03ab-442a-8b81-846260fbd05e\") " May 27 21:39:16.411104 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:39:16.411027 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tp78t\" (UniqueName: \"kubernetes.io/projected/71c6995f-03ab-442a-8b81-846260fbd05e-kube-api-access-tp78t\") pod \"71c6995f-03ab-442a-8b81-846260fbd05e\" (UID: \"71c6995f-03ab-442a-8b81-846260fbd05e\") " May 27 21:39:16.411215 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:39:16.411201 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqhgq\" (UniqueName: \"kubernetes.io/projected/dc9d433c-4d58-439e-8633-fcfff628ab03-kube-api-access-cqhgq\") pod \"limitador-operator-controller-manager-85c4996f8c-pt7th\" (UID: \"dc9d433c-4d58-439e-8633-fcfff628ab03\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-pt7th" May 27 21:39:16.411255 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:39:16.411226 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71c6995f-03ab-442a-8b81-846260fbd05e-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "71c6995f-03ab-442a-8b81-846260fbd05e" (UID: "71c6995f-03ab-442a-8b81-846260fbd05e"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 27 21:39:16.413041 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:39:16.413021 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71c6995f-03ab-442a-8b81-846260fbd05e-kube-api-access-tp78t" (OuterVolumeSpecName: "kube-api-access-tp78t") pod "71c6995f-03ab-442a-8b81-846260fbd05e" (UID: "71c6995f-03ab-442a-8b81-846260fbd05e"). InnerVolumeSpecName "kube-api-access-tp78t". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 27 21:39:16.512512 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:39:16.512452 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cqhgq\" (UniqueName: \"kubernetes.io/projected/dc9d433c-4d58-439e-8633-fcfff628ab03-kube-api-access-cqhgq\") pod \"limitador-operator-controller-manager-85c4996f8c-pt7th\" (UID: \"dc9d433c-4d58-439e-8633-fcfff628ab03\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-pt7th" May 27 21:39:16.512612 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:39:16.512523 2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/71c6995f-03ab-442a-8b81-846260fbd05e-extensions-socket-volume\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:39:16.512612 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:39:16.512532 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tp78t\" (UniqueName: \"kubernetes.io/projected/71c6995f-03ab-442a-8b81-846260fbd05e-kube-api-access-tp78t\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:39:16.525524 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:39:16.525498 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqhgq\" (UniqueName: \"kubernetes.io/projected/dc9d433c-4d58-439e-8633-fcfff628ab03-kube-api-access-cqhgq\") pod \"limitador-operator-controller-manager-85c4996f8c-pt7th\" (UID: \"dc9d433c-4d58-439e-8633-fcfff628ab03\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-pt7th" May 27 21:39:16.603550 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:39:16.603526 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-pt7th" May 27 21:39:16.744042 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:39:16.744011 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-pt7th"] May 27 21:39:16.745086 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:39:16.745056 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc9d433c_4d58_439e_8633_fcfff628ab03.slice/crio-f60c50b53bf510cc3e1c6b19f2a3be357eaf1da633f8072f4d36de1fa84f40f1 WatchSource:0}: Error finding container f60c50b53bf510cc3e1c6b19f2a3be357eaf1da633f8072f4d36de1fa84f40f1: Status 404 returned error can't find the container with id f60c50b53bf510cc3e1c6b19f2a3be357eaf1da633f8072f4d36de1fa84f40f1 May 27 21:39:16.746848 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:39:16.746831 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 27 21:39:17.083923 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:39:17.083893 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71c6995f-03ab-442a-8b81-846260fbd05e" path="/var/lib/kubelet/pods/71c6995f-03ab-442a-8b81-846260fbd05e/volumes" May 27 21:39:17.086221 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:39:17.086191 2564 status_manager.go:895] "Failed to get status for pod" podUID="71c6995f-03ab-442a-8b81-846260fbd05e" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-pqxpq" err="pods \"kuadrant-operator-controller-manager-84b657d985-pqxpq\" is forbidden: User \"system:node:ip-10-0-130-246.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-246.ec2.internal' and this object" May 27 21:39:17.383360 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:39:17.383260 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-pt7th" event={"ID":"dc9d433c-4d58-439e-8633-fcfff628ab03","Type":"ContainerStarted","Data":"f60c50b53bf510cc3e1c6b19f2a3be357eaf1da633f8072f4d36de1fa84f40f1"} May 27 21:39:17.383501 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:39:17.383369 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-pqxpq" May 27 21:39:17.388869 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:39:17.388827 2564 status_manager.go:895] "Failed to get status for pod" podUID="71c6995f-03ab-442a-8b81-846260fbd05e" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-pqxpq" err="pods \"kuadrant-operator-controller-manager-84b657d985-pqxpq\" is forbidden: User \"system:node:ip-10-0-130-246.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-246.ec2.internal' and this object" May 27 21:39:19.392804 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:39:19.392761 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-pt7th" event={"ID":"dc9d433c-4d58-439e-8633-fcfff628ab03","Type":"ContainerStarted","Data":"e4f89002827b2d0cb1a1f70412ce586b4f57b00df5ee222061db042729395d42"} May 27 21:39:19.393222 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:39:19.392875 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-pt7th" May 27 21:39:19.423590 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:39:19.423540 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-pt7th" podStartSLOduration=1.802844742 podStartE2EDuration="3.423527527s" podCreationTimestamp="2026-05-27 21:39:16 +0000 UTC" firstStartedPulling="2026-05-27 21:39:16.746957548 +0000 UTC m=+630.221162255" lastFinishedPulling="2026-05-27 21:39:18.367640321 +0000 UTC m=+631.841845040" observedRunningTime="2026-05-27 21:39:19.423085515 +0000 UTC m=+632.897290244" watchObservedRunningTime="2026-05-27 21:39:19.423527527 +0000 UTC m=+632.897732255" May 27 21:39:30.398559 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:39:30.398525 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-pt7th" May 27 21:40:28.245759 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:40:28.245725 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-c987dfc98-gmr9w"] May 27 21:40:28.248517 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:40:28.248497 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c987dfc98-gmr9w" May 27 21:40:28.251278 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:40:28.251243 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" May 27 21:40:28.251379 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:40:28.251330 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-9zqvr\"" May 27 21:40:28.257730 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:40:28.257707 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-c987dfc98-gmr9w"] May 27 21:40:28.322130 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:40:28.322096 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtllc\" (UniqueName: \"kubernetes.io/projected/58f2355e-03c6-4344-9808-9abc4e7322c5-kube-api-access-dtllc\") pod \"maas-controller-c987dfc98-gmr9w\" (UID: \"58f2355e-03c6-4344-9808-9abc4e7322c5\") " pod="opendatahub/maas-controller-c987dfc98-gmr9w" May 27 21:40:28.398794 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:40:28.398760 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-66545d889b-mjdlb"] May 27 21:40:28.413286 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:40:28.401865 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-66545d889b-mjdlb" May 27 21:40:28.413286 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:40:28.412176 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-66545d889b-mjdlb"] May 27 21:40:28.422784 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:40:28.422751 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dtllc\" (UniqueName: \"kubernetes.io/projected/58f2355e-03c6-4344-9808-9abc4e7322c5-kube-api-access-dtllc\") pod \"maas-controller-c987dfc98-gmr9w\" (UID: \"58f2355e-03c6-4344-9808-9abc4e7322c5\") " pod="opendatahub/maas-controller-c987dfc98-gmr9w" May 27 21:40:28.431560 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:40:28.431533 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtllc\" (UniqueName: \"kubernetes.io/projected/58f2355e-03c6-4344-9808-9abc4e7322c5-kube-api-access-dtllc\") pod \"maas-controller-c987dfc98-gmr9w\" (UID: \"58f2355e-03c6-4344-9808-9abc4e7322c5\") " pod="opendatahub/maas-controller-c987dfc98-gmr9w" May 27 21:40:28.526324 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:40:28.526209 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bk268\" (UniqueName: \"kubernetes.io/projected/d2f667a9-a736-4c3e-875f-1f02ac39bb56-kube-api-access-bk268\") pod \"maas-controller-66545d889b-mjdlb\" (UID: \"d2f667a9-a736-4c3e-875f-1f02ac39bb56\") " pod="opendatahub/maas-controller-66545d889b-mjdlb" May 27 21:40:28.559331 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:40:28.559305 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c987dfc98-gmr9w" May 27 21:40:28.626916 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:40:28.626884 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bk268\" (UniqueName: \"kubernetes.io/projected/d2f667a9-a736-4c3e-875f-1f02ac39bb56-kube-api-access-bk268\") pod \"maas-controller-66545d889b-mjdlb\" (UID: \"d2f667a9-a736-4c3e-875f-1f02ac39bb56\") " pod="opendatahub/maas-controller-66545d889b-mjdlb" May 27 21:40:28.636012 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:40:28.635989 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bk268\" (UniqueName: \"kubernetes.io/projected/d2f667a9-a736-4c3e-875f-1f02ac39bb56-kube-api-access-bk268\") pod \"maas-controller-66545d889b-mjdlb\" (UID: \"d2f667a9-a736-4c3e-875f-1f02ac39bb56\") " pod="opendatahub/maas-controller-66545d889b-mjdlb" May 27 21:40:28.690780 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:40:28.690749 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-c987dfc98-gmr9w"] May 27 21:40:28.691814 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:40:28.691786 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58f2355e_03c6_4344_9808_9abc4e7322c5.slice/crio-2b709ac6592b9220146702b5e2559f2b60adea0931cbd0beadb6a39dd776aa51 WatchSource:0}: Error finding container 2b709ac6592b9220146702b5e2559f2b60adea0931cbd0beadb6a39dd776aa51: Status 404 returned error can't find the container with id 2b709ac6592b9220146702b5e2559f2b60adea0931cbd0beadb6a39dd776aa51 May 27 21:40:28.721220 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:40:28.721187 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-66545d889b-mjdlb" May 27 21:40:28.851953 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:40:28.851928 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-66545d889b-mjdlb"] May 27 21:40:28.853405 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:40:28.853381 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2f667a9_a736_4c3e_875f_1f02ac39bb56.slice/crio-4a80d53af656472f1fae2b036168c7843d114fabb1f178b40602562e656949b3 WatchSource:0}: Error finding container 4a80d53af656472f1fae2b036168c7843d114fabb1f178b40602562e656949b3: Status 404 returned error can't find the container with id 4a80d53af656472f1fae2b036168c7843d114fabb1f178b40602562e656949b3 May 27 21:40:29.667339 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:40:29.667300 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c987dfc98-gmr9w" event={"ID":"58f2355e-03c6-4344-9808-9abc4e7322c5","Type":"ContainerStarted","Data":"2b709ac6592b9220146702b5e2559f2b60adea0931cbd0beadb6a39dd776aa51"} May 27 21:40:29.669745 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:40:29.669717 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-66545d889b-mjdlb" event={"ID":"d2f667a9-a736-4c3e-875f-1f02ac39bb56","Type":"ContainerStarted","Data":"4a80d53af656472f1fae2b036168c7843d114fabb1f178b40602562e656949b3"} May 27 21:40:31.683006 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:40:31.682958 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c987dfc98-gmr9w" event={"ID":"58f2355e-03c6-4344-9808-9abc4e7322c5","Type":"ContainerStarted","Data":"2041bc0d2485b67c172f774ad4c1fe95c4df762fe966343de9f90ba491530ac5"} May 27 21:40:31.683470 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:40:31.683177 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-c987dfc98-gmr9w" May 27 21:40:31.684400 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:40:31.684377 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-66545d889b-mjdlb" event={"ID":"d2f667a9-a736-4c3e-875f-1f02ac39bb56","Type":"ContainerStarted","Data":"be8aa2f29e597062ed476699706bc50111dd83e23116dc06ad150505837bbeeb"} May 27 21:40:31.684565 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:40:31.684546 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-66545d889b-mjdlb" May 27 21:40:31.751342 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:40:31.751289 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-66545d889b-mjdlb" podStartSLOduration=1.21149905 podStartE2EDuration="3.751256099s" podCreationTimestamp="2026-05-27 21:40:28 +0000 UTC" firstStartedPulling="2026-05-27 21:40:28.854712249 +0000 UTC m=+702.328916956" lastFinishedPulling="2026-05-27 21:40:31.394469285 +0000 UTC m=+704.868674005" observedRunningTime="2026-05-27 21:40:31.750449654 +0000 UTC m=+705.224654382" watchObservedRunningTime="2026-05-27 21:40:31.751256099 +0000 UTC m=+705.225460828" May 27 21:40:31.752503 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:40:31.752465 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-c987dfc98-gmr9w" podStartSLOduration=1.06166951 podStartE2EDuration="3.752450555s" podCreationTimestamp="2026-05-27 21:40:28 +0000 UTC" firstStartedPulling="2026-05-27 21:40:28.693067515 +0000 UTC m=+702.167272223" lastFinishedPulling="2026-05-27 21:40:31.383848556 +0000 UTC m=+704.858053268" observedRunningTime="2026-05-27 21:40:31.719240333 +0000 UTC m=+705.193445063" watchObservedRunningTime="2026-05-27 21:40:31.752450555 +0000 UTC m=+705.226655284" May 27 21:40:42.694886 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:40:42.694802 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-66545d889b-mjdlb" May 27 21:40:42.695354 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:40:42.695068 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-c987dfc98-gmr9w" May 27 21:40:42.761238 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:40:42.761200 2564 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-c987dfc98-gmr9w"] May 27 21:40:42.761531 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:40:42.761506 2564 kuberuntime_container.go:864] "Killing container with a grace period" pod="opendatahub/maas-controller-c987dfc98-gmr9w" podUID="58f2355e-03c6-4344-9808-9abc4e7322c5" containerName="manager" containerID="cri-o://2041bc0d2485b67c172f774ad4c1fe95c4df762fe966343de9f90ba491530ac5" gracePeriod=30 May 27 21:40:43.004907 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:40:43.004884 2564 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c987dfc98-gmr9w" May 27 21:40:43.158425 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:40:43.158388 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dtllc\" (UniqueName: \"kubernetes.io/projected/58f2355e-03c6-4344-9808-9abc4e7322c5-kube-api-access-dtllc\") pod \"58f2355e-03c6-4344-9808-9abc4e7322c5\" (UID: \"58f2355e-03c6-4344-9808-9abc4e7322c5\") " May 27 21:40:43.160540 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:40:43.160514 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58f2355e-03c6-4344-9808-9abc4e7322c5-kube-api-access-dtllc" (OuterVolumeSpecName: "kube-api-access-dtllc") pod "58f2355e-03c6-4344-9808-9abc4e7322c5" (UID: "58f2355e-03c6-4344-9808-9abc4e7322c5"). InnerVolumeSpecName "kube-api-access-dtllc". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 27 21:40:43.259478 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:40:43.259392 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dtllc\" (UniqueName: \"kubernetes.io/projected/58f2355e-03c6-4344-9808-9abc4e7322c5-kube-api-access-dtllc\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:40:43.739785 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:40:43.739748 2564 generic.go:358] "Generic (PLEG): container finished" podID="58f2355e-03c6-4344-9808-9abc4e7322c5" containerID="2041bc0d2485b67c172f774ad4c1fe95c4df762fe966343de9f90ba491530ac5" exitCode=0 May 27 21:40:43.740228 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:40:43.739808 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c987dfc98-gmr9w" event={"ID":"58f2355e-03c6-4344-9808-9abc4e7322c5","Type":"ContainerDied","Data":"2041bc0d2485b67c172f774ad4c1fe95c4df762fe966343de9f90ba491530ac5"} May 27 21:40:43.740228 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:40:43.739825 2564 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c987dfc98-gmr9w" May 27 21:40:43.740228 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:40:43.739856 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c987dfc98-gmr9w" event={"ID":"58f2355e-03c6-4344-9808-9abc4e7322c5","Type":"ContainerDied","Data":"2b709ac6592b9220146702b5e2559f2b60adea0931cbd0beadb6a39dd776aa51"} May 27 21:40:43.740228 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:40:43.739878 2564 scope.go:117] "RemoveContainer" containerID="2041bc0d2485b67c172f774ad4c1fe95c4df762fe966343de9f90ba491530ac5" May 27 21:40:43.749479 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:40:43.749459 2564 scope.go:117] "RemoveContainer" containerID="2041bc0d2485b67c172f774ad4c1fe95c4df762fe966343de9f90ba491530ac5" May 27 21:40:43.749766 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:40:43.749741 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2041bc0d2485b67c172f774ad4c1fe95c4df762fe966343de9f90ba491530ac5\": container with ID starting with 2041bc0d2485b67c172f774ad4c1fe95c4df762fe966343de9f90ba491530ac5 not found: ID does not exist" containerID="2041bc0d2485b67c172f774ad4c1fe95c4df762fe966343de9f90ba491530ac5" May 27 21:40:43.749834 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:40:43.749780 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2041bc0d2485b67c172f774ad4c1fe95c4df762fe966343de9f90ba491530ac5"} err="failed to get container status \"2041bc0d2485b67c172f774ad4c1fe95c4df762fe966343de9f90ba491530ac5\": rpc error: code = NotFound desc = could not find container \"2041bc0d2485b67c172f774ad4c1fe95c4df762fe966343de9f90ba491530ac5\": container with ID starting with 2041bc0d2485b67c172f774ad4c1fe95c4df762fe966343de9f90ba491530ac5 not found: ID does not exist" May 27 21:40:43.763297 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:40:43.763260 2564 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-c987dfc98-gmr9w"] May 27 21:40:43.767767 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:40:43.767743 2564 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-c987dfc98-gmr9w"] May 27 21:40:45.084329 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:40:45.084294 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58f2355e-03c6-4344-9808-9abc4e7322c5" path="/var/lib/kubelet/pods/58f2355e-03c6-4344-9808-9abc4e7322c5/volumes" May 27 21:41:03.140601 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:41:03.140566 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58b6c8fdc7-4bnbb"] May 27 21:41:03.141053 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:41:03.140941 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58f2355e-03c6-4344-9808-9abc4e7322c5" containerName="manager" May 27 21:41:03.141053 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:41:03.140956 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="58f2355e-03c6-4344-9808-9abc4e7322c5" containerName="manager" May 27 21:41:03.141053 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:41:03.141018 2564 memory_manager.go:356] "RemoveStaleState removing state" podUID="58f2355e-03c6-4344-9808-9abc4e7322c5" containerName="manager" May 27 21:41:03.145384 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:41:03.145368 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58b6c8fdc7-4bnbb" May 27 21:41:03.148873 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:41:03.148793 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-6fvf2\"" May 27 21:41:03.149328 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:41:03.149292 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" May 27 21:41:03.157714 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:41:03.157692 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58b6c8fdc7-4bnbb"] May 27 21:41:03.222636 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:41:03.222596 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9qm2\" (UniqueName: \"kubernetes.io/projected/a68c7dbb-f5a3-4308-a98b-d67dedad1cc0-kube-api-access-l9qm2\") pod \"payload-processing-58b6c8fdc7-4bnbb\" (UID: \"a68c7dbb-f5a3-4308-a98b-d67dedad1cc0\") " pod="openshift-ingress/payload-processing-58b6c8fdc7-4bnbb" May 27 21:41:03.323029 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:41:03.322988 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-l9qm2\" (UniqueName: \"kubernetes.io/projected/a68c7dbb-f5a3-4308-a98b-d67dedad1cc0-kube-api-access-l9qm2\") pod \"payload-processing-58b6c8fdc7-4bnbb\" (UID: \"a68c7dbb-f5a3-4308-a98b-d67dedad1cc0\") " pod="openshift-ingress/payload-processing-58b6c8fdc7-4bnbb" May 27 21:41:03.332976 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:41:03.332941 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9qm2\" (UniqueName: \"kubernetes.io/projected/a68c7dbb-f5a3-4308-a98b-d67dedad1cc0-kube-api-access-l9qm2\") pod \"payload-processing-58b6c8fdc7-4bnbb\" (UID: \"a68c7dbb-f5a3-4308-a98b-d67dedad1cc0\") " pod="openshift-ingress/payload-processing-58b6c8fdc7-4bnbb" May 27 21:41:03.455805 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:41:03.455765 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58b6c8fdc7-4bnbb" May 27 21:41:03.592700 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:41:03.592672 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58b6c8fdc7-4bnbb"] May 27 21:41:03.593898 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:41:03.593873 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda68c7dbb_f5a3_4308_a98b_d67dedad1cc0.slice/crio-ee57dca8034fa7afe34dd58a744724024d779e256d925957e24378b21d3f0705 WatchSource:0}: Error finding container ee57dca8034fa7afe34dd58a744724024d779e256d925957e24378b21d3f0705: Status 404 returned error can't find the container with id ee57dca8034fa7afe34dd58a744724024d779e256d925957e24378b21d3f0705 May 27 21:41:03.827667 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:41:03.827576 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58b6c8fdc7-4bnbb" event={"ID":"a68c7dbb-f5a3-4308-a98b-d67dedad1cc0","Type":"ContainerStarted","Data":"ee57dca8034fa7afe34dd58a744724024d779e256d925957e24378b21d3f0705"} May 27 21:41:05.836675 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:41:05.836638 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58b6c8fdc7-4bnbb" event={"ID":"a68c7dbb-f5a3-4308-a98b-d67dedad1cc0","Type":"ContainerStarted","Data":"7abdf58437a02b69b4d5cb51a50af91ecd5b3693907a212fe4f1e2975bba23c3"} May 27 21:41:05.837106 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:41:05.836753 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58b6c8fdc7-4bnbb" May 27 21:41:05.854066 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:41:05.854001 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58b6c8fdc7-4bnbb" podStartSLOduration=1.2716340320000001 podStartE2EDuration="2.853982952s" podCreationTimestamp="2026-05-27 21:41:03 +0000 UTC" firstStartedPulling="2026-05-27 21:41:03.595636763 +0000 UTC m=+737.069841470" lastFinishedPulling="2026-05-27 21:41:05.177985683 +0000 UTC m=+738.652190390" observedRunningTime="2026-05-27 21:41:05.852749532 +0000 UTC m=+739.326954260" watchObservedRunningTime="2026-05-27 21:41:05.853982952 +0000 UTC m=+739.328187682" May 27 21:41:16.843491 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:41:16.843456 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58b6c8fdc7-4bnbb" May 27 21:41:50.203226 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:41:50.203189 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-xvvgz"] May 27 21:41:50.207064 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:41:50.207041 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-xvvgz" May 27 21:41:50.210013 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:41:50.209994 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" May 27 21:41:50.211068 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:41:50.211046 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" May 27 21:41:50.211175 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:41:50.211081 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-v28hb\"" May 27 21:41:50.211175 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:41:50.211096 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" May 27 21:41:50.222612 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:41:50.222590 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-xvvgz"] May 27 21:41:50.225845 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:41:50.225823 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b8c2a083-9947-4636-bbc8-05dc27c11908-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-xvvgz\" (UID: \"b8c2a083-9947-4636-bbc8-05dc27c11908\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-xvvgz" May 27 21:41:50.225938 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:41:50.225867 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmvbv\" (UniqueName: \"kubernetes.io/projected/b8c2a083-9947-4636-bbc8-05dc27c11908-kube-api-access-zmvbv\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-xvvgz\" (UID: \"b8c2a083-9947-4636-bbc8-05dc27c11908\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-xvvgz" May 27 21:41:50.225977 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:41:50.225934 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b8c2a083-9947-4636-bbc8-05dc27c11908-home\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-xvvgz\" (UID: \"b8c2a083-9947-4636-bbc8-05dc27c11908\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-xvvgz" May 27 21:41:50.226012 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:41:50.225976 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b8c2a083-9947-4636-bbc8-05dc27c11908-model-cache\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-xvvgz\" (UID: \"b8c2a083-9947-4636-bbc8-05dc27c11908\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-xvvgz" May 27 21:41:50.226012 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:41:50.226003 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b8c2a083-9947-4636-bbc8-05dc27c11908-dshm\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-xvvgz\" (UID: \"b8c2a083-9947-4636-bbc8-05dc27c11908\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-xvvgz" May 27 21:41:50.226081 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:41:50.226070 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b8c2a083-9947-4636-bbc8-05dc27c11908-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-xvvgz\" (UID: \"b8c2a083-9947-4636-bbc8-05dc27c11908\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-xvvgz" May 27 21:41:50.326740 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:41:50.326702 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b8c2a083-9947-4636-bbc8-05dc27c11908-home\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-xvvgz\" (UID: \"b8c2a083-9947-4636-bbc8-05dc27c11908\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-xvvgz" May 27 21:41:50.326740 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:41:50.326745 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b8c2a083-9947-4636-bbc8-05dc27c11908-model-cache\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-xvvgz\" (UID: \"b8c2a083-9947-4636-bbc8-05dc27c11908\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-xvvgz" May 27 21:41:50.326974 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:41:50.326845 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b8c2a083-9947-4636-bbc8-05dc27c11908-dshm\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-xvvgz\" (UID: \"b8c2a083-9947-4636-bbc8-05dc27c11908\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-xvvgz" May 27 21:41:50.326974 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:41:50.326894 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b8c2a083-9947-4636-bbc8-05dc27c11908-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-xvvgz\" (UID: \"b8c2a083-9947-4636-bbc8-05dc27c11908\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-xvvgz" May 27 21:41:50.326974 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:41:50.326919 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b8c2a083-9947-4636-bbc8-05dc27c11908-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-xvvgz\" (UID: \"b8c2a083-9947-4636-bbc8-05dc27c11908\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-xvvgz" May 27 21:41:50.326974 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:41:50.326944 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zmvbv\" (UniqueName: \"kubernetes.io/projected/b8c2a083-9947-4636-bbc8-05dc27c11908-kube-api-access-zmvbv\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-xvvgz\" (UID: \"b8c2a083-9947-4636-bbc8-05dc27c11908\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-xvvgz" May 27 21:41:50.327181 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:41:50.327163 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b8c2a083-9947-4636-bbc8-05dc27c11908-home\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-xvvgz\" (UID: \"b8c2a083-9947-4636-bbc8-05dc27c11908\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-xvvgz" May 27 21:41:50.327227 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:41:50.327185 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b8c2a083-9947-4636-bbc8-05dc27c11908-model-cache\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-xvvgz\" (UID: \"b8c2a083-9947-4636-bbc8-05dc27c11908\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-xvvgz" May 27 21:41:50.327290 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:41:50.327235 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b8c2a083-9947-4636-bbc8-05dc27c11908-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-xvvgz\" (UID: \"b8c2a083-9947-4636-bbc8-05dc27c11908\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-xvvgz" May 27 21:41:50.329091 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:41:50.329062 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b8c2a083-9947-4636-bbc8-05dc27c11908-dshm\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-xvvgz\" (UID: \"b8c2a083-9947-4636-bbc8-05dc27c11908\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-xvvgz" May 27 21:41:50.329340 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:41:50.329323 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b8c2a083-9947-4636-bbc8-05dc27c11908-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-xvvgz\" (UID: \"b8c2a083-9947-4636-bbc8-05dc27c11908\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-xvvgz" May 27 21:41:50.336945 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:41:50.336920 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmvbv\" (UniqueName: \"kubernetes.io/projected/b8c2a083-9947-4636-bbc8-05dc27c11908-kube-api-access-zmvbv\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-xvvgz\" (UID: \"b8c2a083-9947-4636-bbc8-05dc27c11908\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-xvvgz" May 27 21:41:50.517230 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:41:50.517143 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-xvvgz" May 27 21:41:50.657714 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:41:50.657688 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-xvvgz"] May 27 21:41:50.658662 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:41:50.658625 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8c2a083_9947_4636_bbc8_05dc27c11908.slice/crio-ca60820af8ae27d52529baabaeb08bbeaccde405c71dfa28edd1f737cb28bac3 WatchSource:0}: Error finding container ca60820af8ae27d52529baabaeb08bbeaccde405c71dfa28edd1f737cb28bac3: Status 404 returned error can't find the container with id ca60820af8ae27d52529baabaeb08bbeaccde405c71dfa28edd1f737cb28bac3 May 27 21:41:51.014121 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:41:51.014084 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-xvvgz" event={"ID":"b8c2a083-9947-4636-bbc8-05dc27c11908","Type":"ContainerStarted","Data":"ca60820af8ae27d52529baabaeb08bbeaccde405c71dfa28edd1f737cb28bac3"} May 27 21:41:57.041813 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:41:57.041773 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-xvvgz" event={"ID":"b8c2a083-9947-4636-bbc8-05dc27c11908","Type":"ContainerStarted","Data":"53116cc199340a2efad40ccb5a588f97ea59e061448d6e63735a082ba1d19ac1"} May 27 21:42:02.062175 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:42:02.062139 2564 generic.go:358] "Generic (PLEG): container finished" podID="b8c2a083-9947-4636-bbc8-05dc27c11908" containerID="53116cc199340a2efad40ccb5a588f97ea59e061448d6e63735a082ba1d19ac1" exitCode=0 May 27 21:42:02.062558 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:42:02.062217 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-xvvgz" event={"ID":"b8c2a083-9947-4636-bbc8-05dc27c11908","Type":"ContainerDied","Data":"53116cc199340a2efad40ccb5a588f97ea59e061448d6e63735a082ba1d19ac1"} May 27 21:42:04.075414 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:42:04.075376 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-xvvgz" event={"ID":"b8c2a083-9947-4636-bbc8-05dc27c11908","Type":"ContainerStarted","Data":"41faca961040200b14058b56c8017dfe9f030e4bdc1f286e180af26c23dc79d1"} May 27 21:42:04.075816 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:42:04.075625 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-xvvgz" May 27 21:42:04.097120 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:42:04.097066 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-xvvgz" podStartSLOduration=1.594171606 podStartE2EDuration="14.097051797s" podCreationTimestamp="2026-05-27 21:41:50 +0000 UTC" firstStartedPulling="2026-05-27 21:41:50.660465847 +0000 UTC m=+784.134670554" lastFinishedPulling="2026-05-27 21:42:03.163346034 +0000 UTC m=+796.637550745" observedRunningTime="2026-05-27 21:42:04.094921838 +0000 UTC m=+797.569126580" watchObservedRunningTime="2026-05-27 21:42:04.097051797 +0000 UTC m=+797.571256527" May 27 21:42:15.093453 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:42:15.093371 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-xvvgz" May 27 21:43:47.057739 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:43:47.057638 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-d9gcd_16b8cf1e-1181-43fc-92e0-635b8523248d/console-operator/2.log" May 27 21:43:47.059014 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:43:47.058993 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-d9gcd_16b8cf1e-1181-43fc-92e0-635b8523248d/console-operator/2.log" May 27 21:44:06.746848 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:44:06.746815 2564 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-66545d889b-mjdlb"] May 27 21:44:06.747584 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:44:06.747060 2564 kuberuntime_container.go:864] "Killing container with a grace period" pod="opendatahub/maas-controller-66545d889b-mjdlb" podUID="d2f667a9-a736-4c3e-875f-1f02ac39bb56" containerName="manager" containerID="cri-o://be8aa2f29e597062ed476699706bc50111dd83e23116dc06ad150505837bbeeb" gracePeriod=30 May 27 21:44:06.990867 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:44:06.990844 2564 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-66545d889b-mjdlb" May 27 21:44:07.118563 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:44:07.118492 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bk268\" (UniqueName: \"kubernetes.io/projected/d2f667a9-a736-4c3e-875f-1f02ac39bb56-kube-api-access-bk268\") pod \"d2f667a9-a736-4c3e-875f-1f02ac39bb56\" (UID: \"d2f667a9-a736-4c3e-875f-1f02ac39bb56\") " May 27 21:44:07.120414 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:44:07.120389 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2f667a9-a736-4c3e-875f-1f02ac39bb56-kube-api-access-bk268" (OuterVolumeSpecName: "kube-api-access-bk268") pod "d2f667a9-a736-4c3e-875f-1f02ac39bb56" (UID: "d2f667a9-a736-4c3e-875f-1f02ac39bb56"). InnerVolumeSpecName "kube-api-access-bk268". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 27 21:44:07.219545 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:44:07.219508 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bk268\" (UniqueName: \"kubernetes.io/projected/d2f667a9-a736-4c3e-875f-1f02ac39bb56-kube-api-access-bk268\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:44:07.553625 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:44:07.553594 2564 generic.go:358] "Generic (PLEG): container finished" podID="d2f667a9-a736-4c3e-875f-1f02ac39bb56" containerID="be8aa2f29e597062ed476699706bc50111dd83e23116dc06ad150505837bbeeb" exitCode=0 May 27 21:44:07.553811 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:44:07.553685 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-66545d889b-mjdlb" event={"ID":"d2f667a9-a736-4c3e-875f-1f02ac39bb56","Type":"ContainerDied","Data":"be8aa2f29e597062ed476699706bc50111dd83e23116dc06ad150505837bbeeb"} May 27 21:44:07.553811 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:44:07.553704 2564 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-66545d889b-mjdlb" May 27 21:44:07.553811 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:44:07.553726 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-66545d889b-mjdlb" event={"ID":"d2f667a9-a736-4c3e-875f-1f02ac39bb56","Type":"ContainerDied","Data":"4a80d53af656472f1fae2b036168c7843d114fabb1f178b40602562e656949b3"} May 27 21:44:07.553811 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:44:07.553741 2564 scope.go:117] "RemoveContainer" containerID="be8aa2f29e597062ed476699706bc50111dd83e23116dc06ad150505837bbeeb" May 27 21:44:07.564076 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:44:07.564013 2564 scope.go:117] "RemoveContainer" containerID="be8aa2f29e597062ed476699706bc50111dd83e23116dc06ad150505837bbeeb" May 27 21:44:07.564967 ip-10-0-130-246 kubenswrapper[2564]: E0527 21:44:07.564937 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be8aa2f29e597062ed476699706bc50111dd83e23116dc06ad150505837bbeeb\": container with ID starting with be8aa2f29e597062ed476699706bc50111dd83e23116dc06ad150505837bbeeb not found: ID does not exist" containerID="be8aa2f29e597062ed476699706bc50111dd83e23116dc06ad150505837bbeeb" May 27 21:44:07.565068 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:44:07.564977 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be8aa2f29e597062ed476699706bc50111dd83e23116dc06ad150505837bbeeb"} err="failed to get container status \"be8aa2f29e597062ed476699706bc50111dd83e23116dc06ad150505837bbeeb\": rpc error: code = NotFound desc = could not find container \"be8aa2f29e597062ed476699706bc50111dd83e23116dc06ad150505837bbeeb\": container with ID starting with be8aa2f29e597062ed476699706bc50111dd83e23116dc06ad150505837bbeeb not found: ID does not exist" May 27 21:44:07.584221 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:44:07.584195 2564 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-66545d889b-mjdlb"] May 27 21:44:07.586442 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:44:07.586422 2564 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-66545d889b-mjdlb"] May 27 21:44:08.341075 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:44:08.341038 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-66545d889b-x6jkg"] May 27 21:44:08.341455 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:44:08.341444 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2f667a9-a736-4c3e-875f-1f02ac39bb56" containerName="manager" May 27 21:44:08.341495 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:44:08.341458 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2f667a9-a736-4c3e-875f-1f02ac39bb56" containerName="manager" May 27 21:44:08.341528 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:44:08.341510 2564 memory_manager.go:356] "RemoveStaleState removing state" podUID="d2f667a9-a736-4c3e-875f-1f02ac39bb56" containerName="manager" May 27 21:44:08.345800 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:44:08.345781 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-66545d889b-x6jkg" May 27 21:44:08.348429 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:44:08.348397 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" May 27 21:44:08.349492 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:44:08.349472 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-9zqvr\"" May 27 21:44:08.353187 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:44:08.353164 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-66545d889b-x6jkg"] May 27 21:44:08.431837 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:44:08.431807 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpf5t\" (UniqueName: \"kubernetes.io/projected/6fb6f549-c45c-4c91-ab50-9187a3dcabe6-kube-api-access-wpf5t\") pod \"maas-controller-66545d889b-x6jkg\" (UID: \"6fb6f549-c45c-4c91-ab50-9187a3dcabe6\") " pod="opendatahub/maas-controller-66545d889b-x6jkg" May 27 21:44:08.532535 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:44:08.532502 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wpf5t\" (UniqueName: \"kubernetes.io/projected/6fb6f549-c45c-4c91-ab50-9187a3dcabe6-kube-api-access-wpf5t\") pod \"maas-controller-66545d889b-x6jkg\" (UID: \"6fb6f549-c45c-4c91-ab50-9187a3dcabe6\") " pod="opendatahub/maas-controller-66545d889b-x6jkg" May 27 21:44:08.543826 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:44:08.543793 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpf5t\" (UniqueName: \"kubernetes.io/projected/6fb6f549-c45c-4c91-ab50-9187a3dcabe6-kube-api-access-wpf5t\") pod \"maas-controller-66545d889b-x6jkg\" (UID: \"6fb6f549-c45c-4c91-ab50-9187a3dcabe6\") " pod="opendatahub/maas-controller-66545d889b-x6jkg" May 27 21:44:08.657070 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:44:08.656988 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-66545d889b-x6jkg" May 27 21:44:08.804274 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:44:08.804239 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-66545d889b-x6jkg"] May 27 21:44:08.806471 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:44:08.806437 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fb6f549_c45c_4c91_ab50_9187a3dcabe6.slice/crio-585e7eeadcf1e56f13a890a54ca2c57a0c4ef02d3e1c612e81eef8799aa4680b WatchSource:0}: Error finding container 585e7eeadcf1e56f13a890a54ca2c57a0c4ef02d3e1c612e81eef8799aa4680b: Status 404 returned error can't find the container with id 585e7eeadcf1e56f13a890a54ca2c57a0c4ef02d3e1c612e81eef8799aa4680b May 27 21:44:09.084384 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:44:09.084350 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2f667a9-a736-4c3e-875f-1f02ac39bb56" path="/var/lib/kubelet/pods/d2f667a9-a736-4c3e-875f-1f02ac39bb56/volumes" May 27 21:44:09.564298 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:44:09.564241 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-66545d889b-x6jkg" event={"ID":"6fb6f549-c45c-4c91-ab50-9187a3dcabe6","Type":"ContainerStarted","Data":"56e373bc9ddd09d9d167ce3ec4f5eaa6846a434357925621ca85ba31f8e160e1"} May 27 21:44:09.564298 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:44:09.564296 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-66545d889b-x6jkg" event={"ID":"6fb6f549-c45c-4c91-ab50-9187a3dcabe6","Type":"ContainerStarted","Data":"585e7eeadcf1e56f13a890a54ca2c57a0c4ef02d3e1c612e81eef8799aa4680b"} May 27 21:44:09.564770 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:44:09.564332 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-66545d889b-x6jkg" May 27 21:44:09.587546 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:44:09.587489 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-66545d889b-x6jkg" podStartSLOduration=1.148414047 podStartE2EDuration="1.587475627s" podCreationTimestamp="2026-05-27 21:44:08 +0000 UTC" firstStartedPulling="2026-05-27 21:44:08.807743236 +0000 UTC m=+922.281947943" lastFinishedPulling="2026-05-27 21:44:09.2468048 +0000 UTC m=+922.721009523" observedRunningTime="2026-05-27 21:44:09.584714139 +0000 UTC m=+923.058918905" watchObservedRunningTime="2026-05-27 21:44:09.587475627 +0000 UTC m=+923.061680394" May 27 21:44:20.575091 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:44:20.575051 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-66545d889b-x6jkg" May 27 21:45:00.143559 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:45:00.143518 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29665305-s6bsn"] May 27 21:45:00.148197 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:45:00.148176 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29665305-s6bsn" May 27 21:45:00.151111 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:45:00.151088 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-9wzcg\"" May 27 21:45:00.158371 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:45:00.158341 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29665305-s6bsn"] May 27 21:45:00.246800 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:45:00.246768 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5txn8\" (UniqueName: \"kubernetes.io/projected/5a1e15f7-3080-4c87-90f9-b7bc7ab8c4f0-kube-api-access-5txn8\") pod \"maas-api-key-cleanup-29665305-s6bsn\" (UID: \"5a1e15f7-3080-4c87-90f9-b7bc7ab8c4f0\") " pod="opendatahub/maas-api-key-cleanup-29665305-s6bsn" May 27 21:45:00.348084 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:45:00.348043 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5txn8\" (UniqueName: \"kubernetes.io/projected/5a1e15f7-3080-4c87-90f9-b7bc7ab8c4f0-kube-api-access-5txn8\") pod \"maas-api-key-cleanup-29665305-s6bsn\" (UID: \"5a1e15f7-3080-4c87-90f9-b7bc7ab8c4f0\") " pod="opendatahub/maas-api-key-cleanup-29665305-s6bsn" May 27 21:45:00.357435 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:45:00.357408 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5txn8\" (UniqueName: \"kubernetes.io/projected/5a1e15f7-3080-4c87-90f9-b7bc7ab8c4f0-kube-api-access-5txn8\") pod \"maas-api-key-cleanup-29665305-s6bsn\" (UID: \"5a1e15f7-3080-4c87-90f9-b7bc7ab8c4f0\") " pod="opendatahub/maas-api-key-cleanup-29665305-s6bsn" May 27 21:45:00.459058 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:45:00.459034 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29665305-s6bsn" May 27 21:45:00.793749 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:45:00.793722 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29665305-s6bsn"] May 27 21:45:00.795340 ip-10-0-130-246 kubenswrapper[2564]: W0527 21:45:00.795309 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a1e15f7_3080_4c87_90f9_b7bc7ab8c4f0.slice/crio-6082d39ce5b995ddad38e3da6cecf1929da704d9799713096d39ad6a272f3701 WatchSource:0}: Error finding container 6082d39ce5b995ddad38e3da6cecf1929da704d9799713096d39ad6a272f3701: Status 404 returned error can't find the container with id 6082d39ce5b995ddad38e3da6cecf1929da704d9799713096d39ad6a272f3701 May 27 21:45:00.797119 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:45:00.797102 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 27 21:45:01.778815 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:45:01.778781 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29665305-s6bsn" event={"ID":"5a1e15f7-3080-4c87-90f9-b7bc7ab8c4f0","Type":"ContainerStarted","Data":"6082d39ce5b995ddad38e3da6cecf1929da704d9799713096d39ad6a272f3701"} May 27 21:45:03.792213 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:45:03.792168 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29665305-s6bsn" event={"ID":"5a1e15f7-3080-4c87-90f9-b7bc7ab8c4f0","Type":"ContainerStarted","Data":"2500a71148dd860c62f6a174a9616063f56f66882978ca9a2af300ec03eddf30"} May 27 21:45:03.808901 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:45:03.808846 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29665305-s6bsn" podStartSLOduration=1.297136704 podStartE2EDuration="3.808820664s" podCreationTimestamp="2026-05-27 21:45:00 +0000 UTC" firstStartedPulling="2026-05-27 21:45:00.797226587 +0000 UTC m=+974.271431295" lastFinishedPulling="2026-05-27 21:45:03.308910537 +0000 UTC m=+976.783115255" observedRunningTime="2026-05-27 21:45:03.80786672 +0000 UTC m=+977.282071440" watchObservedRunningTime="2026-05-27 21:45:03.808820664 +0000 UTC m=+977.283025372" May 27 21:45:04.797489 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:45:04.797447 2564 generic.go:358] "Generic (PLEG): container finished" podID="5a1e15f7-3080-4c87-90f9-b7bc7ab8c4f0" containerID="2500a71148dd860c62f6a174a9616063f56f66882978ca9a2af300ec03eddf30" exitCode=0 May 27 21:45:04.797887 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:45:04.797540 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29665305-s6bsn" event={"ID":"5a1e15f7-3080-4c87-90f9-b7bc7ab8c4f0","Type":"ContainerDied","Data":"2500a71148dd860c62f6a174a9616063f56f66882978ca9a2af300ec03eddf30"} May 27 21:45:05.931060 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:45:05.931036 2564 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29665305-s6bsn" May 27 21:45:06.000448 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:45:06.000415 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5txn8\" (UniqueName: \"kubernetes.io/projected/5a1e15f7-3080-4c87-90f9-b7bc7ab8c4f0-kube-api-access-5txn8\") pod \"5a1e15f7-3080-4c87-90f9-b7bc7ab8c4f0\" (UID: \"5a1e15f7-3080-4c87-90f9-b7bc7ab8c4f0\") " May 27 21:45:06.002509 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:45:06.002486 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a1e15f7-3080-4c87-90f9-b7bc7ab8c4f0-kube-api-access-5txn8" (OuterVolumeSpecName: "kube-api-access-5txn8") pod "5a1e15f7-3080-4c87-90f9-b7bc7ab8c4f0" (UID: "5a1e15f7-3080-4c87-90f9-b7bc7ab8c4f0"). InnerVolumeSpecName "kube-api-access-5txn8". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 27 21:45:06.101391 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:45:06.101301 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5txn8\" (UniqueName: \"kubernetes.io/projected/5a1e15f7-3080-4c87-90f9-b7bc7ab8c4f0-kube-api-access-5txn8\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 21:45:06.808560 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:45:06.808520 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29665305-s6bsn" event={"ID":"5a1e15f7-3080-4c87-90f9-b7bc7ab8c4f0","Type":"ContainerDied","Data":"6082d39ce5b995ddad38e3da6cecf1929da704d9799713096d39ad6a272f3701"} May 27 21:45:06.808560 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:45:06.808564 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6082d39ce5b995ddad38e3da6cecf1929da704d9799713096d39ad6a272f3701" May 27 21:45:06.808755 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:45:06.808537 2564 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29665305-s6bsn" May 27 21:48:47.092898 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:48:47.092824 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-d9gcd_16b8cf1e-1181-43fc-92e0-635b8523248d/console-operator/2.log" May 27 21:48:47.093903 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:48:47.093883 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-d9gcd_16b8cf1e-1181-43fc-92e0-635b8523248d/console-operator/2.log" May 27 21:53:47.127419 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:53:47.127389 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-d9gcd_16b8cf1e-1181-43fc-92e0-635b8523248d/console-operator/2.log" May 27 21:53:47.129148 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:53:47.129121 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-d9gcd_16b8cf1e-1181-43fc-92e0-635b8523248d/console-operator/2.log" May 27 21:58:47.166954 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:58:47.166922 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-d9gcd_16b8cf1e-1181-43fc-92e0-635b8523248d/console-operator/2.log" May 27 21:58:47.169915 ip-10-0-130-246 kubenswrapper[2564]: I0527 21:58:47.169891 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-d9gcd_16b8cf1e-1181-43fc-92e0-635b8523248d/console-operator/2.log" May 27 22:00:00.136572 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:00:00.136525 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29665320-5zjd5"] May 27 22:00:00.137165 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:00:00.137110 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a1e15f7-3080-4c87-90f9-b7bc7ab8c4f0" containerName="cleanup" May 27 22:00:00.137165 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:00:00.137131 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a1e15f7-3080-4c87-90f9-b7bc7ab8c4f0" containerName="cleanup" May 27 22:00:00.137309 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:00:00.137216 2564 memory_manager.go:356] "RemoveStaleState removing state" podUID="5a1e15f7-3080-4c87-90f9-b7bc7ab8c4f0" containerName="cleanup" May 27 22:00:00.140322 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:00:00.140299 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29665320-5zjd5" May 27 22:00:00.142706 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:00:00.142687 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-9wzcg\"" May 27 22:00:00.157731 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:00:00.157706 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29665320-5zjd5"] May 27 22:00:00.302514 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:00:00.302484 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5w62\" (UniqueName: \"kubernetes.io/projected/9151bf3c-abcc-4764-a56f-fb32aaea955e-kube-api-access-m5w62\") pod \"maas-api-key-cleanup-29665320-5zjd5\" (UID: \"9151bf3c-abcc-4764-a56f-fb32aaea955e\") " pod="opendatahub/maas-api-key-cleanup-29665320-5zjd5" May 27 22:00:00.403469 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:00:00.403393 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-m5w62\" (UniqueName: \"kubernetes.io/projected/9151bf3c-abcc-4764-a56f-fb32aaea955e-kube-api-access-m5w62\") pod \"maas-api-key-cleanup-29665320-5zjd5\" (UID: \"9151bf3c-abcc-4764-a56f-fb32aaea955e\") " pod="opendatahub/maas-api-key-cleanup-29665320-5zjd5" May 27 22:00:00.413296 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:00:00.413255 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5w62\" (UniqueName: \"kubernetes.io/projected/9151bf3c-abcc-4764-a56f-fb32aaea955e-kube-api-access-m5w62\") pod \"maas-api-key-cleanup-29665320-5zjd5\" (UID: \"9151bf3c-abcc-4764-a56f-fb32aaea955e\") " pod="opendatahub/maas-api-key-cleanup-29665320-5zjd5" May 27 22:00:00.450699 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:00:00.450675 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29665320-5zjd5" May 27 22:00:00.791829 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:00:00.791798 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29665320-5zjd5"] May 27 22:00:00.793467 ip-10-0-130-246 kubenswrapper[2564]: W0527 22:00:00.793435 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9151bf3c_abcc_4764_a56f_fb32aaea955e.slice/crio-d59d60ed478c491eaa7d22e6df8c8aebe6a94bb9b9b55e2d2f845146410fbe2f WatchSource:0}: Error finding container d59d60ed478c491eaa7d22e6df8c8aebe6a94bb9b9b55e2d2f845146410fbe2f: Status 404 returned error can't find the container with id d59d60ed478c491eaa7d22e6df8c8aebe6a94bb9b9b55e2d2f845146410fbe2f May 27 22:00:00.795369 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:00:00.795354 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 27 22:00:01.332331 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:00:01.332293 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29665320-5zjd5" event={"ID":"9151bf3c-abcc-4764-a56f-fb32aaea955e","Type":"ContainerStarted","Data":"60e9733ff4b3b34eaa14bc716548c880fac622353879172bf5d67f8b36d0bccb"} May 27 22:00:01.332331 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:00:01.332336 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29665320-5zjd5" event={"ID":"9151bf3c-abcc-4764-a56f-fb32aaea955e","Type":"ContainerStarted","Data":"d59d60ed478c491eaa7d22e6df8c8aebe6a94bb9b9b55e2d2f845146410fbe2f"} May 27 22:00:01.354673 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:00:01.354627 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29665320-5zjd5" podStartSLOduration=1.354613379 podStartE2EDuration="1.354613379s" podCreationTimestamp="2026-05-27 22:00:00 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-27 22:00:01.352445193 +0000 UTC m=+1874.826649923" watchObservedRunningTime="2026-05-27 22:00:01.354613379 +0000 UTC m=+1874.828818108" May 27 22:00:02.339058 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:00:02.339024 2564 generic.go:358] "Generic (PLEG): container finished" podID="9151bf3c-abcc-4764-a56f-fb32aaea955e" containerID="60e9733ff4b3b34eaa14bc716548c880fac622353879172bf5d67f8b36d0bccb" exitCode=0 May 27 22:00:02.339497 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:00:02.339116 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29665320-5zjd5" event={"ID":"9151bf3c-abcc-4764-a56f-fb32aaea955e","Type":"ContainerDied","Data":"60e9733ff4b3b34eaa14bc716548c880fac622353879172bf5d67f8b36d0bccb"} May 27 22:00:03.482074 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:00:03.482046 2564 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29665320-5zjd5" May 27 22:00:03.636508 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:00:03.636423 2564 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m5w62\" (UniqueName: \"kubernetes.io/projected/9151bf3c-abcc-4764-a56f-fb32aaea955e-kube-api-access-m5w62\") pod \"9151bf3c-abcc-4764-a56f-fb32aaea955e\" (UID: \"9151bf3c-abcc-4764-a56f-fb32aaea955e\") " May 27 22:00:03.638620 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:00:03.638582 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9151bf3c-abcc-4764-a56f-fb32aaea955e-kube-api-access-m5w62" (OuterVolumeSpecName: "kube-api-access-m5w62") pod "9151bf3c-abcc-4764-a56f-fb32aaea955e" (UID: "9151bf3c-abcc-4764-a56f-fb32aaea955e"). InnerVolumeSpecName "kube-api-access-m5w62". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 27 22:00:03.737978 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:00:03.737935 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m5w62\" (UniqueName: \"kubernetes.io/projected/9151bf3c-abcc-4764-a56f-fb32aaea955e-kube-api-access-m5w62\") on node \"ip-10-0-130-246.ec2.internal\" DevicePath \"\"" May 27 22:00:04.348210 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:00:04.348175 2564 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29665320-5zjd5" May 27 22:00:04.348419 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:00:04.348178 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29665320-5zjd5" event={"ID":"9151bf3c-abcc-4764-a56f-fb32aaea955e","Type":"ContainerDied","Data":"d59d60ed478c491eaa7d22e6df8c8aebe6a94bb9b9b55e2d2f845146410fbe2f"} May 27 22:00:04.348419 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:00:04.348303 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d59d60ed478c491eaa7d22e6df8c8aebe6a94bb9b9b55e2d2f845146410fbe2f" May 27 22:03:47.201283 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:03:47.201231 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-d9gcd_16b8cf1e-1181-43fc-92e0-635b8523248d/console-operator/2.log" May 27 22:03:47.205162 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:03:47.205141 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-d9gcd_16b8cf1e-1181-43fc-92e0-635b8523248d/console-operator/2.log" May 27 22:05:21.879803 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:21.879771 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-856948b99f-vjsc4_7ef19ce3-6eb8-4064-9c5e-31eb60b8c56a/manager/0.log" May 27 22:05:22.113530 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:22.113502 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29665305-s6bsn_5a1e15f7-3080-4c87-90f9-b7bc7ab8c4f0/cleanup/0.log" May 27 22:05:22.216254 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:22.216226 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29665320-5zjd5_9151bf3c-abcc-4764-a56f-fb32aaea955e/cleanup/0.log" May 27 22:05:22.329956 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:22.329930 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-66545d889b-x6jkg_6fb6f549-c45c-4c91-ab50-9187a3dcabe6/manager/0.log" May 27 22:05:22.447387 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:22.447351 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-858dbf95b8-tn94s_6333b9d0-8455-4556-adde-669e2259a722/manager/2.log" May 27 22:05:22.677069 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:22.677038 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7d799877b9-bjwnb_db87c04f-749b-4491-8b16-7c072e2714f9/manager/0.log" May 27 22:05:23.690259 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:23.690225 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75975psw_844b76fb-f722-4de9-a98d-fb539fead6f0/util/0.log" May 27 22:05:23.699786 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:23.699757 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75975psw_844b76fb-f722-4de9-a98d-fb539fead6f0/pull/0.log" May 27 22:05:23.707500 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:23.707472 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75975psw_844b76fb-f722-4de9-a98d-fb539fead6f0/extract/0.log" May 27 22:05:23.817723 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:23.817697 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xvgn2_f1233a43-b5ac-45a9-96e7-56b99df116b7/util/0.log" May 27 22:05:23.826814 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:23.826791 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xvgn2_f1233a43-b5ac-45a9-96e7-56b99df116b7/pull/0.log" May 27 22:05:23.834970 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:23.834936 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xvgn2_f1233a43-b5ac-45a9-96e7-56b99df116b7/extract/0.log" May 27 22:05:23.949238 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:23.949170 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dv7fr_bad5a47a-1b3c-4c5b-88a3-b5972403452f/util/0.log" May 27 22:05:23.961305 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:23.961258 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dv7fr_bad5a47a-1b3c-4c5b-88a3-b5972403452f/pull/0.log" May 27 22:05:23.973097 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:23.973079 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dv7fr_bad5a47a-1b3c-4c5b-88a3-b5972403452f/extract/0.log" May 27 22:05:24.089730 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:24.089704 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tqm2b_8515de3f-c993-4bd8-bb41-91ec6d3bbf2b/extract/0.log" May 27 22:05:24.102918 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:24.102889 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tqm2b_8515de3f-c993-4bd8-bb41-91ec6d3bbf2b/util/0.log" May 27 22:05:24.111854 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:24.111834 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tqm2b_8515de3f-c993-4bd8-bb41-91ec6d3bbf2b/pull/0.log" May 27 22:05:24.358786 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:24.358711 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-657f44b778-lznn9_b3cc5b72-1ec8-4ebe-b7bf-9c9ae0be8e3a/manager/0.log" May 27 22:05:25.078075 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:25.078043 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-85c4996f8c-pt7th_dc9d433c-4d58-439e-8633-fcfff628ab03/manager/0.log" May 27 22:05:25.536620 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:25.536592 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-798958bb55-pbr96_b04535f7-68a3-4874-af75-5ca190852e78/discovery/0.log" May 27 22:05:25.753364 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:25.753337 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-68b96d7ccc-ghzfh_f1965d5f-a8e4-4a0b-9e0b-4596c90ab37f/kube-auth-proxy/0.log" May 27 22:05:25.988671 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:25.988646 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58b6c8fdc7-4bnbb_a68c7dbb-f5a3-4308-a98b-d67dedad1cc0/payload-processing/0.log" May 27 22:05:26.095784 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:26.095755 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-688f8b8554-fn4zx_51660acb-022f-4265-bec8-ae93f5315b17/router/0.log" May 27 22:05:26.893411 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:26.893383 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-8f8dc67b7-xvvgz_b8c2a083-9947-4636-bbc8-05dc27c11908/storage-initializer/0.log" May 27 22:05:26.900946 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:26.900922 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-8f8dc67b7-xvvgz_b8c2a083-9947-4636-bbc8-05dc27c11908/main/0.log" May 27 22:05:34.203506 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:34.203467 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-vm9vc_2174f1a7-3c35-461b-a60d-b2b6714e35f8/global-pull-secret-syncer/0.log" May 27 22:05:34.305888 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:34.305862 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-p6vj8_657348d2-3465-4003-9ef6-3139dc8c1a92/konnectivity-agent/0.log" May 27 22:05:34.356013 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:34.355987 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-246.ec2.internal_6614026242b9bed0ac8a71fe40323c9f/haproxy/0.log" May 27 22:05:38.017236 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:38.017208 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75975psw_844b76fb-f722-4de9-a98d-fb539fead6f0/extract/0.log" May 27 22:05:38.046789 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:38.046753 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75975psw_844b76fb-f722-4de9-a98d-fb539fead6f0/util/0.log" May 27 22:05:38.074751 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:38.074727 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75975psw_844b76fb-f722-4de9-a98d-fb539fead6f0/pull/0.log" May 27 22:05:38.103482 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:38.103460 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xvgn2_f1233a43-b5ac-45a9-96e7-56b99df116b7/extract/0.log" May 27 22:05:38.125030 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:38.125007 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xvgn2_f1233a43-b5ac-45a9-96e7-56b99df116b7/util/0.log" May 27 22:05:38.152329 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:38.152310 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xvgn2_f1233a43-b5ac-45a9-96e7-56b99df116b7/pull/0.log" May 27 22:05:38.181158 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:38.181131 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dv7fr_bad5a47a-1b3c-4c5b-88a3-b5972403452f/extract/0.log" May 27 22:05:38.244599 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:38.244575 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dv7fr_bad5a47a-1b3c-4c5b-88a3-b5972403452f/util/0.log" May 27 22:05:38.271721 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:38.271657 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dv7fr_bad5a47a-1b3c-4c5b-88a3-b5972403452f/pull/0.log" May 27 22:05:38.298167 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:38.298147 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tqm2b_8515de3f-c993-4bd8-bb41-91ec6d3bbf2b/extract/0.log" May 27 22:05:38.319464 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:38.319444 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tqm2b_8515de3f-c993-4bd8-bb41-91ec6d3bbf2b/util/0.log" May 27 22:05:38.344390 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:38.344374 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tqm2b_8515de3f-c993-4bd8-bb41-91ec6d3bbf2b/pull/0.log" May 27 22:05:38.695390 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:38.695357 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-657f44b778-lznn9_b3cc5b72-1ec8-4ebe-b7bf-9c9ae0be8e3a/manager/0.log" May 27 22:05:39.031645 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:39.031565 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-85c4996f8c-pt7th_dc9d433c-4d58-439e-8633-fcfff628ab03/manager/0.log" May 27 22:05:40.787841 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:40.787769 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-54fd6bc5c4-77nkf_030b9490-f0b8-4a41-90a9-da20af4103be/kube-state-metrics/0.log" May 27 22:05:40.814904 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:40.814881 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-54fd6bc5c4-77nkf_030b9490-f0b8-4a41-90a9-da20af4103be/kube-rbac-proxy-main/0.log" May 27 22:05:40.837397 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:40.837381 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-54fd6bc5c4-77nkf_030b9490-f0b8-4a41-90a9-da20af4103be/kube-rbac-proxy-self/0.log" May 27 22:05:40.890220 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:40.890198 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-665c9c8584-xshg4_b7a8fe62-9758-4449-b720-78d9c0842165/monitoring-plugin/0.log" May 27 22:05:41.005768 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:41.005746 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8prph_32f3075a-ac99-468a-a2b9-c76f16897682/node-exporter/0.log" May 27 22:05:41.041097 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:41.041030 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8prph_32f3075a-ac99-468a-a2b9-c76f16897682/kube-rbac-proxy/0.log" May 27 22:05:41.076251 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:41.076229 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8prph_32f3075a-ac99-468a-a2b9-c76f16897682/init-textfile/0.log" May 27 22:05:41.475310 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:41.475286 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7779bbdd78-jhdrv_ece3e9ae-4d45-45d6-a435-df9cb81ebb42/prometheus-operator/0.log" May 27 22:05:41.496705 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:41.496673 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7779bbdd78-jhdrv_ece3e9ae-4d45-45d6-a435-df9cb81ebb42/kube-rbac-proxy/0.log" May 27 22:05:41.527174 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:41.527149 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-6c4bbc67b7-jq28q_4da4d84a-f34f-44d3-9312-a8888ac22fa3/prometheus-operator-admission-webhook/0.log" May 27 22:05:43.111032 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:43.110999 2564 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-77md2/perf-node-gather-daemonset-gsfhh"] May 27 22:05:43.111461 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:43.111444 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9151bf3c-abcc-4764-a56f-fb32aaea955e" containerName="cleanup" May 27 22:05:43.111461 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:43.111463 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9151bf3c-abcc-4764-a56f-fb32aaea955e" containerName="cleanup" May 27 22:05:43.111572 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:43.111544 2564 memory_manager.go:356] "RemoveStaleState removing state" podUID="9151bf3c-abcc-4764-a56f-fb32aaea955e" containerName="cleanup" May 27 22:05:43.113652 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:43.113633 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-77md2/perf-node-gather-daemonset-gsfhh" May 27 22:05:43.116405 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:43.116377 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-77md2\"/\"kube-root-ca.crt\"" May 27 22:05:43.116498 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:43.116408 2564 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-77md2\"/\"openshift-service-ca.crt\"" May 27 22:05:43.117320 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:43.117302 2564 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-77md2\"/\"default-dockercfg-84qd4\"" May 27 22:05:43.126398 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:43.126377 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-77md2/perf-node-gather-daemonset-gsfhh"] May 27 22:05:43.266380 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:43.266345 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4494aeae-9616-46b5-a22c-9423adc1f91d-sys\") pod \"perf-node-gather-daemonset-gsfhh\" (UID: \"4494aeae-9616-46b5-a22c-9423adc1f91d\") " pod="openshift-must-gather-77md2/perf-node-gather-daemonset-gsfhh" May 27 22:05:43.266553 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:43.266395 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4494aeae-9616-46b5-a22c-9423adc1f91d-lib-modules\") pod \"perf-node-gather-daemonset-gsfhh\" (UID: \"4494aeae-9616-46b5-a22c-9423adc1f91d\") " pod="openshift-must-gather-77md2/perf-node-gather-daemonset-gsfhh" May 27 22:05:43.266553 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:43.266471 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9zjz\" (UniqueName: \"kubernetes.io/projected/4494aeae-9616-46b5-a22c-9423adc1f91d-kube-api-access-b9zjz\") pod \"perf-node-gather-daemonset-gsfhh\" (UID: \"4494aeae-9616-46b5-a22c-9423adc1f91d\") " pod="openshift-must-gather-77md2/perf-node-gather-daemonset-gsfhh" May 27 22:05:43.266682 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:43.266554 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4494aeae-9616-46b5-a22c-9423adc1f91d-podres\") pod \"perf-node-gather-daemonset-gsfhh\" (UID: \"4494aeae-9616-46b5-a22c-9423adc1f91d\") " pod="openshift-must-gather-77md2/perf-node-gather-daemonset-gsfhh" May 27 22:05:43.266682 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:43.266608 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4494aeae-9616-46b5-a22c-9423adc1f91d-proc\") pod \"perf-node-gather-daemonset-gsfhh\" (UID: \"4494aeae-9616-46b5-a22c-9423adc1f91d\") " pod="openshift-must-gather-77md2/perf-node-gather-daemonset-gsfhh" May 27 22:05:43.367206 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:43.367102 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4494aeae-9616-46b5-a22c-9423adc1f91d-podres\") pod \"perf-node-gather-daemonset-gsfhh\" (UID: \"4494aeae-9616-46b5-a22c-9423adc1f91d\") " pod="openshift-must-gather-77md2/perf-node-gather-daemonset-gsfhh" May 27 22:05:43.367206 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:43.367161 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4494aeae-9616-46b5-a22c-9423adc1f91d-proc\") pod \"perf-node-gather-daemonset-gsfhh\" (UID: \"4494aeae-9616-46b5-a22c-9423adc1f91d\") " pod="openshift-must-gather-77md2/perf-node-gather-daemonset-gsfhh" May 27 22:05:43.367206 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:43.367205 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4494aeae-9616-46b5-a22c-9423adc1f91d-sys\") pod \"perf-node-gather-daemonset-gsfhh\" (UID: \"4494aeae-9616-46b5-a22c-9423adc1f91d\") " pod="openshift-must-gather-77md2/perf-node-gather-daemonset-gsfhh" May 27 22:05:43.367520 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:43.367242 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4494aeae-9616-46b5-a22c-9423adc1f91d-lib-modules\") pod \"perf-node-gather-daemonset-gsfhh\" (UID: \"4494aeae-9616-46b5-a22c-9423adc1f91d\") " pod="openshift-must-gather-77md2/perf-node-gather-daemonset-gsfhh" May 27 22:05:43.367520 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:43.367332 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4494aeae-9616-46b5-a22c-9423adc1f91d-proc\") pod \"perf-node-gather-daemonset-gsfhh\" (UID: \"4494aeae-9616-46b5-a22c-9423adc1f91d\") " pod="openshift-must-gather-77md2/perf-node-gather-daemonset-gsfhh" May 27 22:05:43.367520 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:43.367332 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4494aeae-9616-46b5-a22c-9423adc1f91d-sys\") pod \"perf-node-gather-daemonset-gsfhh\" (UID: \"4494aeae-9616-46b5-a22c-9423adc1f91d\") " pod="openshift-must-gather-77md2/perf-node-gather-daemonset-gsfhh" May 27 22:05:43.367520 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:43.367334 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4494aeae-9616-46b5-a22c-9423adc1f91d-podres\") pod \"perf-node-gather-daemonset-gsfhh\" (UID: \"4494aeae-9616-46b5-a22c-9423adc1f91d\") " pod="openshift-must-gather-77md2/perf-node-gather-daemonset-gsfhh" May 27 22:05:43.367520 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:43.367339 2564 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-b9zjz\" (UniqueName: \"kubernetes.io/projected/4494aeae-9616-46b5-a22c-9423adc1f91d-kube-api-access-b9zjz\") pod \"perf-node-gather-daemonset-gsfhh\" (UID: \"4494aeae-9616-46b5-a22c-9423adc1f91d\") " pod="openshift-must-gather-77md2/perf-node-gather-daemonset-gsfhh" May 27 22:05:43.367520 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:43.367408 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4494aeae-9616-46b5-a22c-9423adc1f91d-lib-modules\") pod \"perf-node-gather-daemonset-gsfhh\" (UID: \"4494aeae-9616-46b5-a22c-9423adc1f91d\") " pod="openshift-must-gather-77md2/perf-node-gather-daemonset-gsfhh" May 27 22:05:43.377130 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:43.377102 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9zjz\" (UniqueName: \"kubernetes.io/projected/4494aeae-9616-46b5-a22c-9423adc1f91d-kube-api-access-b9zjz\") pod \"perf-node-gather-daemonset-gsfhh\" (UID: \"4494aeae-9616-46b5-a22c-9423adc1f91d\") " pod="openshift-must-gather-77md2/perf-node-gather-daemonset-gsfhh" May 27 22:05:43.398601 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:43.398580 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-d9gcd_16b8cf1e-1181-43fc-92e0-635b8523248d/console-operator/2.log" May 27 22:05:43.402819 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:43.402803 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-d9gcd_16b8cf1e-1181-43fc-92e0-635b8523248d/console-operator/3.log" May 27 22:05:43.424256 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:43.424237 2564 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-77md2/perf-node-gather-daemonset-gsfhh" May 27 22:05:43.561954 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:43.561926 2564 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-77md2/perf-node-gather-daemonset-gsfhh"] May 27 22:05:43.563172 ip-10-0-130-246 kubenswrapper[2564]: W0527 22:05:43.563141 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod4494aeae_9616_46b5_a22c_9423adc1f91d.slice/crio-03f2d1dde43643eec324ef5c2e1a33192975dd61f13c5ac36432e008c1136742 WatchSource:0}: Error finding container 03f2d1dde43643eec324ef5c2e1a33192975dd61f13c5ac36432e008c1136742: Status 404 returned error can't find the container with id 03f2d1dde43643eec324ef5c2e1a33192975dd61f13c5ac36432e008c1136742 May 27 22:05:43.564906 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:43.564890 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 27 22:05:43.702676 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:43.702644 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-77md2/perf-node-gather-daemonset-gsfhh" event={"ID":"4494aeae-9616-46b5-a22c-9423adc1f91d","Type":"ContainerStarted","Data":"47d394a993c5a572206bdb734262b1a04aa80379bca1198799f3c4c4f6ade37c"} May 27 22:05:43.702676 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:43.702681 2564 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-77md2/perf-node-gather-daemonset-gsfhh" event={"ID":"4494aeae-9616-46b5-a22c-9423adc1f91d","Type":"ContainerStarted","Data":"03f2d1dde43643eec324ef5c2e1a33192975dd61f13c5ac36432e008c1136742"} May 27 22:05:43.702884 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:43.702696 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-77md2/perf-node-gather-daemonset-gsfhh" May 27 22:05:43.717469 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:43.717420 2564 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-77md2/perf-node-gather-daemonset-gsfhh" podStartSLOduration=0.717404793 podStartE2EDuration="717.404793ms" podCreationTimestamp="2026-05-27 22:05:43 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-27 22:05:43.716768373 +0000 UTC m=+2217.190973104" watchObservedRunningTime="2026-05-27 22:05:43.717404793 +0000 UTC m=+2217.191609527" May 27 22:05:43.946727 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:43.946703 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-65fcc74dd8-6wspm_038f205d-8fa0-4849-a4f3-bfef23617a86/console/0.log" May 27 22:05:43.980400 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:43.980373 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-57bd96b888-nwvmp_87cf8e90-d191-4ca4-abb0-82f7f557b0e7/download-server/0.log" May 27 22:05:44.497945 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:44.497916 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5cf8c89d64-c5dbz_3bdeafef-3082-4f0e-8add-fcdf60551c63/volume-data-source-validator/0.log" May 27 22:05:45.272784 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:45.272754 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4t29f_f0c8b7dd-e25c-44d5-a1ac-52e4cef19011/dns/0.log" May 27 22:05:45.294467 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:45.294446 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4t29f_f0c8b7dd-e25c-44d5-a1ac-52e4cef19011/kube-rbac-proxy/0.log" May 27 22:05:45.464466 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:45.464437 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-sc6tb_d895055d-d1d4-44a6-9024-8b244e769d5e/dns-node-resolver/0.log" May 27 22:05:45.939557 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:45.939526 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7c87c4fbbc-psmsw_d78abc3c-065a-43d7-a5e0-3cc662c0d057/registry/0.log" May 27 22:05:46.018439 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:46.018409 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mdd25_e176311e-f128-4137-904b-7dd6fea5eed1/node-ca/0.log" May 27 22:05:46.956019 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:46.955989 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-798958bb55-pbr96_b04535f7-68a3-4874-af75-5ca190852e78/discovery/0.log" May 27 22:05:46.996791 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:46.996768 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-68b96d7ccc-ghzfh_f1965d5f-a8e4-4a0b-9e0b-4596c90ab37f/kube-auth-proxy/0.log" May 27 22:05:47.140734 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:47.140706 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58b6c8fdc7-4bnbb_a68c7dbb-f5a3-4308-a98b-d67dedad1cc0/payload-processing/0.log" May 27 22:05:47.174796 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:47.174773 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-688f8b8554-fn4zx_51660acb-022f-4265-bec8-ae93f5315b17/router/0.log" May 27 22:05:47.680495 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:47.680473 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5ntj8_0a23d452-20fc-4665-9d3a-ef17d210a8d9/serve-healthcheck-canary/0.log" May 27 22:05:48.211647 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:48.211617 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7wjhw_aafa4c59-c141-4e77-ae60-d6e48d468223/kube-rbac-proxy/0.log" May 27 22:05:48.231099 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:48.231074 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7wjhw_aafa4c59-c141-4e77-ae60-d6e48d468223/exporter/0.log" May 27 22:05:48.253692 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:48.253671 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7wjhw_aafa4c59-c141-4e77-ae60-d6e48d468223/extractor/0.log" May 27 22:05:49.715908 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:49.715874 2564 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-77md2/perf-node-gather-daemonset-gsfhh" May 27 22:05:50.358810 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:50.358782 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-856948b99f-vjsc4_7ef19ce3-6eb8-4064-9c5e-31eb60b8c56a/manager/0.log" May 27 22:05:50.437943 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:50.437920 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29665305-s6bsn_5a1e15f7-3080-4c87-90f9-b7bc7ab8c4f0/cleanup/0.log" May 27 22:05:50.478660 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:50.478634 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29665320-5zjd5_9151bf3c-abcc-4764-a56f-fb32aaea955e/cleanup/0.log" May 27 22:05:50.560624 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:50.560597 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-66545d889b-x6jkg_6fb6f549-c45c-4c91-ab50-9187a3dcabe6/manager/0.log" May 27 22:05:50.581669 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:50.581647 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-858dbf95b8-tn94s_6333b9d0-8455-4556-adde-669e2259a722/manager/1.log" May 27 22:05:50.592418 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:50.592401 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-858dbf95b8-tn94s_6333b9d0-8455-4556-adde-669e2259a722/manager/2.log" May 27 22:05:50.659979 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:50.659911 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7d799877b9-bjwnb_db87c04f-749b-4491-8b16-7c072e2714f9/manager/0.log" May 27 22:05:56.875846 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:56.875815 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-565644f49-z76cl_1df7cdcd-3a57-4d1a-97d4-2638a9f49055/kube-storage-version-migrator-operator/1.log" May 27 22:05:56.876845 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:56.876823 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-565644f49-z76cl_1df7cdcd-3a57-4d1a-97d4-2638a9f49055/kube-storage-version-migrator-operator/0.log" May 27 22:05:57.905729 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:57.905701 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4k8dw_74672ea1-1457-40d1-b8e1-5acca042ca3e/kube-multus-additional-cni-plugins/0.log" May 27 22:05:57.927176 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:57.927151 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4k8dw_74672ea1-1457-40d1-b8e1-5acca042ca3e/egress-router-binary-copy/0.log" May 27 22:05:57.949240 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:57.949216 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4k8dw_74672ea1-1457-40d1-b8e1-5acca042ca3e/cni-plugins/0.log" May 27 22:05:57.971537 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:57.971509 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4k8dw_74672ea1-1457-40d1-b8e1-5acca042ca3e/bond-cni-plugin/0.log" May 27 22:05:57.992619 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:57.992596 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4k8dw_74672ea1-1457-40d1-b8e1-5acca042ca3e/routeoverride-cni/0.log" May 27 22:05:58.011676 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:58.011656 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4k8dw_74672ea1-1457-40d1-b8e1-5acca042ca3e/whereabouts-cni-bincopy/0.log" May 27 22:05:58.030774 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:58.030754 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4k8dw_74672ea1-1457-40d1-b8e1-5acca042ca3e/whereabouts-cni/0.log" May 27 22:05:58.377296 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:58.377255 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-sm52n_e1f1d6bc-2c10-482e-bfc1-f5a17bcef82a/kube-multus/0.log" May 27 22:05:58.439277 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:58.439235 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-gxt5q_7fc93ab3-b601-4fe7-b073-b8702a09404d/network-metrics-daemon/0.log" May 27 22:05:58.460882 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:58.460859 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-gxt5q_7fc93ab3-b601-4fe7-b073-b8702a09404d/kube-rbac-proxy/0.log" May 27 22:05:59.604901 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:59.604875 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8p7kk_fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5/ovn-controller/0.log" May 27 22:05:59.631579 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:59.631550 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8p7kk_fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5/ovn-acl-logging/0.log" May 27 22:05:59.649373 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:59.649344 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8p7kk_fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5/kube-rbac-proxy-node/0.log" May 27 22:05:59.669055 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:59.669034 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8p7kk_fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5/kube-rbac-proxy-ovn-metrics/0.log" May 27 22:05:59.687581 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:59.687565 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8p7kk_fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5/northd/0.log" May 27 22:05:59.707312 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:59.707292 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8p7kk_fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5/nbdb/0.log" May 27 22:05:59.727059 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:59.727039 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8p7kk_fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5/sbdb/0.log" May 27 22:05:59.827830 ip-10-0-130-246 kubenswrapper[2564]: I0527 22:05:59.827798 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8p7kk_fb58bd7a-94ec-46dc-bc70-5bd3c7a90aa5/ovnkube-controller/0.log"