Jul 13 17:05:01.780370 ip-10-0-136-220 systemd[1]: Starting Kubernetes Kubelet... Jul 13 17:05:02.375712 ip-10-0-136-220 kubenswrapper[2569]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 13 17:05:02.375712 ip-10-0-136-220 kubenswrapper[2569]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 13 17:05:02.375712 ip-10-0-136-220 kubenswrapper[2569]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 13 17:05:02.375712 ip-10-0-136-220 kubenswrapper[2569]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 13 17:05:02.375712 ip-10-0-136-220 kubenswrapper[2569]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 13 17:05:02.375712 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.243406 2569 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 13 17:05:02.375712 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248113 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 17:05:02.375712 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248124 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 17:05:02.375712 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248128 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 17:05:02.375712 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248133 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 17:05:02.375712 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248137 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 17:05:02.375712 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248140 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 17:05:02.375712 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248144 2569 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 17:05:02.375712 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248147 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 17:05:02.325502 ip-10-0-136-220 systemd[1]: Started Kubernetes Kubelet. Jul 13 17:05:02.387597 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248150 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 17:05:02.387597 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248153 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 17:05:02.387597 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248156 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 17:05:02.387597 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248159 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 17:05:02.387597 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248162 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 17:05:02.387597 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248165 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 17:05:02.387597 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248168 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 17:05:02.387597 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248170 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 17:05:02.387597 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248173 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 17:05:02.387597 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248175 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 17:05:02.387597 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248178 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 17:05:02.387597 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248181 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 17:05:02.387597 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248183 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 17:05:02.387597 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248195 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 17:05:02.387597 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248198 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 17:05:02.387597 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248203 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 17:05:02.387597 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248207 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 17:05:02.387597 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248210 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 17:05:02.387597 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248213 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 17:05:02.387597 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248216 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 17:05:02.392729 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248219 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 17:05:02.392729 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248222 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 17:05:02.392729 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248224 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 17:05:02.392729 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248227 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 17:05:02.392729 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248230 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 17:05:02.392729 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248233 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 17:05:02.392729 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248236 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 17:05:02.392729 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248239 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 17:05:02.392729 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248242 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 17:05:02.392729 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248244 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 17:05:02.392729 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248247 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 17:05:02.392729 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248250 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 17:05:02.392729 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248252 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 17:05:02.392729 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248255 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 17:05:02.392729 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248257 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 17:05:02.392729 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248260 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 17:05:02.392729 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248263 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 17:05:02.392729 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248266 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 17:05:02.392729 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248268 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 17:05:02.403502 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248271 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 17:05:02.403502 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248274 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 17:05:02.403502 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248276 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 17:05:02.403502 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248279 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 17:05:02.403502 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248282 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 17:05:02.403502 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248284 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 17:05:02.403502 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248287 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 17:05:02.403502 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248291 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 17:05:02.403502 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248294 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 17:05:02.403502 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248297 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 17:05:02.403502 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248300 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 17:05:02.403502 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248302 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 17:05:02.403502 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248305 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 17:05:02.403502 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248308 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 17:05:02.403502 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248310 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 17:05:02.403502 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248313 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 17:05:02.403502 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248316 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 17:05:02.403502 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248319 2569 feature_gate.go:328] unrecognized feature gate: Example Jul 13 17:05:02.403502 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248321 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 17:05:02.403502 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248324 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 17:05:02.403502 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248328 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 17:05:02.558559 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248332 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 17:05:02.558559 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248336 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 17:05:02.558559 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248339 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 17:05:02.558559 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248342 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 17:05:02.558559 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248345 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 17:05:02.558559 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248348 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 17:05:02.558559 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248352 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 17:05:02.558559 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248355 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 17:05:02.558559 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248358 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 17:05:02.558559 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248361 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 17:05:02.558559 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248364 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 17:05:02.558559 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248367 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 17:05:02.558559 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248370 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 17:05:02.558559 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248372 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 17:05:02.558559 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248375 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 17:05:02.558559 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248378 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 17:05:02.558559 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248381 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 17:05:02.558559 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248384 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 17:05:02.558559 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248386 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 17:05:02.559352 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248389 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 17:05:02.559352 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248392 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 17:05:02.559352 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248394 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 17:05:02.559352 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248397 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 17:05:02.559352 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248399 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 17:05:02.559352 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248402 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 17:05:02.559352 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248404 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 17:05:02.559352 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248407 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 17:05:02.559352 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248409 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 17:05:02.559352 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248412 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 17:05:02.559352 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248415 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 17:05:02.559352 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.248417 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 17:05:02.559352 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249159 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 17:05:02.559352 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249168 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 17:05:02.559352 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249172 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 17:05:02.559352 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249175 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 17:05:02.559352 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249179 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 17:05:02.559352 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249182 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 17:05:02.559352 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249185 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 17:05:02.560369 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249188 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 17:05:02.560369 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249191 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 17:05:02.560369 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249194 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 17:05:02.560369 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249196 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 17:05:02.560369 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249199 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 17:05:02.560369 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249202 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 17:05:02.560369 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249205 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 17:05:02.560369 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249207 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 17:05:02.560369 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249210 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 17:05:02.560369 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249213 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 17:05:02.560369 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249215 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 17:05:02.560369 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249218 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 17:05:02.560369 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249220 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 17:05:02.560369 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249225 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 17:05:02.560369 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249229 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 17:05:02.560369 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249233 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 17:05:02.560369 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249236 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 17:05:02.560369 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249239 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 17:05:02.560369 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249242 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 17:05:02.560369 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249245 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 17:05:02.561366 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249247 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 17:05:02.561366 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249250 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 17:05:02.561366 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249252 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 17:05:02.561366 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249255 2569 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 17:05:02.561366 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249257 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 17:05:02.561366 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249260 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 17:05:02.561366 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249263 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 17:05:02.561366 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249265 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 17:05:02.561366 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249268 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 17:05:02.561366 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249271 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 17:05:02.561366 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249274 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 17:05:02.561366 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249277 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 17:05:02.561366 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249279 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 17:05:02.561366 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249282 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 17:05:02.561366 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249285 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 17:05:02.561366 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249287 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 17:05:02.561366 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249290 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 17:05:02.561366 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249292 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 17:05:02.561366 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249295 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 17:05:02.561366 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249298 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 17:05:02.561987 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249300 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 17:05:02.561987 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249303 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 17:05:02.561987 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249305 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 17:05:02.561987 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249308 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 17:05:02.561987 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249310 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 17:05:02.561987 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249313 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 17:05:02.561987 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249316 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 17:05:02.561987 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249319 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 17:05:02.561987 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249321 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 17:05:02.561987 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249324 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 17:05:02.561987 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249327 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 17:05:02.561987 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249329 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 17:05:02.561987 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249332 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 17:05:02.561987 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249335 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 17:05:02.561987 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249337 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 17:05:02.561987 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249340 2569 feature_gate.go:328] unrecognized feature gate: Example Jul 13 17:05:02.561987 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249344 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 17:05:02.561987 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249347 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 17:05:02.561987 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249350 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 17:05:02.562557 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249353 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 17:05:02.562557 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249356 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 17:05:02.562557 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249359 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 17:05:02.562557 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249362 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 17:05:02.562557 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249364 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 17:05:02.562557 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249370 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 17:05:02.562557 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249372 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 17:05:02.562557 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249375 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 17:05:02.562557 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249377 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 17:05:02.562557 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249380 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 17:05:02.562557 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249385 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 17:05:02.562557 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249388 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 17:05:02.562557 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249392 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 17:05:02.562557 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249395 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 17:05:02.562557 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249398 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 17:05:02.562557 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249401 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 17:05:02.562557 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249404 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 17:05:02.562557 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249407 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 17:05:02.562557 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249410 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 17:05:02.562557 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249412 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 17:05:02.563183 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249415 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 17:05:02.563183 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249418 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 17:05:02.563183 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249421 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 17:05:02.563183 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249423 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 17:05:02.563183 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249426 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 17:05:02.563183 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249428 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 17:05:02.563183 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249431 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 17:05:02.563183 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249433 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 17:05:02.563183 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249436 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 17:05:02.563183 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249439 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 17:05:02.563183 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249441 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 17:05:02.563183 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249444 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 17:05:02.563183 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249447 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 17:05:02.563183 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249800 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 17:05:02.563183 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249806 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 17:05:02.563183 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249808 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 17:05:02.563183 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249811 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 17:05:02.563183 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249816 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 17:05:02.563183 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249821 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 17:05:02.563863 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249824 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 17:05:02.563863 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249826 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 17:05:02.563863 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249829 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 17:05:02.563863 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249831 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 17:05:02.563863 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249834 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 17:05:02.563863 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249836 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 17:05:02.563863 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249839 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 17:05:02.563863 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249842 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 17:05:02.563863 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249845 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 17:05:02.563863 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249847 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 17:05:02.563863 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249850 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 17:05:02.563863 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249853 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 17:05:02.563863 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249855 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 17:05:02.563863 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249858 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 17:05:02.563863 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249861 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 17:05:02.563863 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249864 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 17:05:02.563863 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249866 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 17:05:02.563863 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249869 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 17:05:02.563863 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249872 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 17:05:02.563863 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249874 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 17:05:02.564508 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249877 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 17:05:02.564508 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249880 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 17:05:02.564508 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249882 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 17:05:02.564508 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249885 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 17:05:02.564508 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249887 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 17:05:02.564508 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249890 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 17:05:02.564508 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249892 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 17:05:02.564508 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249895 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 17:05:02.564508 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249897 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 17:05:02.564508 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249900 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 17:05:02.564508 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249905 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 17:05:02.564508 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249909 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 17:05:02.564508 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249911 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 17:05:02.564508 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249914 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 17:05:02.564508 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249916 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 17:05:02.564508 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249919 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 17:05:02.564508 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249921 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 17:05:02.564508 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249924 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 17:05:02.564508 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249927 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 17:05:02.565055 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249929 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 17:05:02.565055 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249932 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 17:05:02.565055 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249934 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 17:05:02.565055 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249937 2569 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 17:05:02.565055 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249941 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 17:05:02.565055 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249943 2569 feature_gate.go:328] unrecognized feature gate: Example Jul 13 17:05:02.565055 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249946 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 17:05:02.565055 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249948 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 17:05:02.565055 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249952 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 17:05:02.565055 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249955 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 17:05:02.565055 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249959 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 17:05:02.565055 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249962 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 17:05:02.565055 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249964 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 17:05:02.565055 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249967 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 17:05:02.565055 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249971 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 17:05:02.565055 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249975 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 17:05:02.565055 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249978 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 17:05:02.565055 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249982 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 17:05:02.565055 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249985 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 17:05:02.565668 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249988 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 17:05:02.565668 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249990 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 17:05:02.565668 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249993 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 17:05:02.565668 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249996 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 17:05:02.565668 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.249999 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 17:05:02.565668 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.250003 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 17:05:02.565668 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.250006 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 17:05:02.565668 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.250009 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 17:05:02.565668 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.250011 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 17:05:02.565668 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.250014 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 17:05:02.565668 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.250017 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 17:05:02.565668 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.250019 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 17:05:02.565668 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.250022 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 17:05:02.565668 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.250024 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 17:05:02.565668 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.250027 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 17:05:02.565668 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.250030 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 17:05:02.565668 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.250032 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 17:05:02.565668 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.250034 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 17:05:02.565668 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.250040 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 17:05:02.565668 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.250043 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 17:05:02.566218 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.250046 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 17:05:02.566218 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.250048 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 17:05:02.566218 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.250051 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 17:05:02.566218 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.250054 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 17:05:02.566218 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.250056 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 17:05:02.566218 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.250059 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 17:05:02.566218 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.250061 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 17:05:02.566218 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.250064 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 17:05:02.566218 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.250067 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 17:05:02.566218 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.250069 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 17:05:02.566218 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.250072 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 17:05:02.566218 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.250089 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 17:05:02.566218 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.250092 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 17:05:02.566218 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.250095 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 17:05:02.566218 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.250097 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 17:05:02.566218 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250167 2569 flags.go:64] FLAG: --address="0.0.0.0" Jul 13 17:05:02.566218 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250176 2569 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 13 17:05:02.566218 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250185 2569 flags.go:64] FLAG: --anonymous-auth="true" Jul 13 17:05:02.566218 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250190 2569 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 13 17:05:02.566218 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250198 2569 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 13 17:05:02.566218 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250202 2569 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 13 17:05:02.566806 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250206 2569 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 13 17:05:02.566806 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250210 2569 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 13 17:05:02.566806 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250214 2569 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 13 17:05:02.566806 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250217 2569 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 13 17:05:02.566806 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250221 2569 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 13 17:05:02.566806 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250224 2569 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 13 17:05:02.566806 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250227 2569 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 13 17:05:02.566806 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250230 2569 flags.go:64] FLAG: --cgroup-root="" Jul 13 17:05:02.566806 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250233 2569 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 13 17:05:02.566806 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250237 2569 flags.go:64] FLAG: --client-ca-file="" Jul 13 17:05:02.566806 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250239 2569 flags.go:64] FLAG: --cloud-provider="external" Jul 13 17:05:02.566806 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250242 2569 flags.go:64] FLAG: --cluster-dns="[]" Jul 13 17:05:02.566806 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250247 2569 flags.go:64] FLAG: --cluster-domain="" Jul 13 17:05:02.566806 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250250 2569 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 13 17:05:02.566806 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250253 2569 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 13 17:05:02.566806 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250256 2569 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 13 17:05:02.566806 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250260 2569 flags.go:64] FLAG: --container-log-max-files="5" Jul 13 17:05:02.566806 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250264 2569 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 13 17:05:02.566806 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250267 2569 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 13 17:05:02.566806 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250269 2569 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 13 17:05:02.566806 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250273 2569 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 13 17:05:02.566806 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250276 2569 flags.go:64] FLAG: --contention-profiling="false" Jul 13 17:05:02.566806 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250279 2569 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 13 17:05:02.566806 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250282 2569 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 13 17:05:02.567438 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250285 2569 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 13 17:05:02.567438 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250288 2569 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 13 17:05:02.567438 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250292 2569 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 13 17:05:02.567438 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250297 2569 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 13 17:05:02.567438 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250301 2569 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 13 17:05:02.567438 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250304 2569 flags.go:64] FLAG: --enable-load-reader="false" Jul 13 17:05:02.567438 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250307 2569 flags.go:64] FLAG: --enable-server="true" Jul 13 17:05:02.567438 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250310 2569 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 13 17:05:02.567438 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250315 2569 flags.go:64] FLAG: --event-burst="100" Jul 13 17:05:02.567438 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250318 2569 flags.go:64] FLAG: --event-qps="50" Jul 13 17:05:02.567438 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250321 2569 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 13 17:05:02.567438 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250324 2569 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 13 17:05:02.567438 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250326 2569 flags.go:64] FLAG: --eviction-hard="" Jul 13 17:05:02.567438 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250330 2569 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 13 17:05:02.567438 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250333 2569 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 13 17:05:02.567438 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250336 2569 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 13 17:05:02.567438 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250339 2569 flags.go:64] FLAG: --eviction-soft="" Jul 13 17:05:02.567438 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250342 2569 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 13 17:05:02.567438 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250345 2569 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 13 17:05:02.567438 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250348 2569 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 13 17:05:02.567438 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250351 2569 flags.go:64] FLAG: --experimental-mounter-path="" Jul 13 17:05:02.567438 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250354 2569 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 13 17:05:02.567438 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250357 2569 flags.go:64] FLAG: --fail-swap-on="true" Jul 13 17:05:02.567438 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250360 2569 flags.go:64] FLAG: --feature-gates="" Jul 13 17:05:02.567438 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250363 2569 flags.go:64] FLAG: --file-check-frequency="20s" Jul 13 17:05:02.568174 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250367 2569 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 13 17:05:02.568174 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250370 2569 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 13 17:05:02.568174 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250381 2569 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 13 17:05:02.568174 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250384 2569 flags.go:64] FLAG: --healthz-port="10248" Jul 13 17:05:02.568174 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250387 2569 flags.go:64] FLAG: --help="false" Jul 13 17:05:02.568174 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250390 2569 flags.go:64] FLAG: --hostname-override="ip-10-0-136-220.ec2.internal" Jul 13 17:05:02.568174 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250393 2569 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 13 17:05:02.568174 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250396 2569 flags.go:64] FLAG: --http-check-frequency="20s" Jul 13 17:05:02.568174 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250399 2569 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 13 17:05:02.568174 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250403 2569 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 13 17:05:02.568174 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250407 2569 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 13 17:05:02.568174 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250412 2569 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 13 17:05:02.568174 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250415 2569 flags.go:64] FLAG: --image-service-endpoint="" Jul 13 17:05:02.568174 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250418 2569 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 13 17:05:02.568174 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250421 2569 flags.go:64] FLAG: --kube-api-burst="100" Jul 13 17:05:02.568174 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250424 2569 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 13 17:05:02.568174 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250427 2569 flags.go:64] FLAG: --kube-api-qps="50" Jul 13 17:05:02.568174 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250430 2569 flags.go:64] FLAG: --kube-reserved="" Jul 13 17:05:02.568174 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250433 2569 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 13 17:05:02.568174 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250436 2569 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 13 17:05:02.568174 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250439 2569 flags.go:64] FLAG: --kubelet-cgroups="" Jul 13 17:05:02.568174 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250442 2569 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 13 17:05:02.568174 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250445 2569 flags.go:64] FLAG: --lock-file="" Jul 13 17:05:02.568174 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250447 2569 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 13 17:05:02.569192 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250450 2569 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 13 17:05:02.569192 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250453 2569 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 13 17:05:02.569192 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250459 2569 flags.go:64] FLAG: --log-json-split-stream="false" Jul 13 17:05:02.569192 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250462 2569 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 13 17:05:02.569192 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250465 2569 flags.go:64] FLAG: --log-text-split-stream="false" Jul 13 17:05:02.569192 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250467 2569 flags.go:64] FLAG: --logging-format="text" Jul 13 17:05:02.569192 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250470 2569 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 13 17:05:02.569192 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250474 2569 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 13 17:05:02.569192 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250477 2569 flags.go:64] FLAG: --manifest-url="" Jul 13 17:05:02.569192 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250480 2569 flags.go:64] FLAG: --manifest-url-header="" Jul 13 17:05:02.569192 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250484 2569 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 13 17:05:02.569192 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250487 2569 flags.go:64] FLAG: --max-open-files="1000000" Jul 13 17:05:02.569192 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250491 2569 flags.go:64] FLAG: --max-pods="110" Jul 13 17:05:02.569192 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250494 2569 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 13 17:05:02.569192 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250497 2569 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 13 17:05:02.569192 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250499 2569 flags.go:64] FLAG: --memory-manager-policy="None" Jul 13 17:05:02.569192 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250503 2569 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 13 17:05:02.569192 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250506 2569 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 13 17:05:02.569192 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250510 2569 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 13 17:05:02.569192 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250514 2569 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 13 17:05:02.569192 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250521 2569 flags.go:64] FLAG: --node-status-max-images="50" Jul 13 17:05:02.569192 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250524 2569 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 13 17:05:02.569192 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250527 2569 flags.go:64] FLAG: --oom-score-adj="-999" Jul 13 17:05:02.569192 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250530 2569 flags.go:64] FLAG: --pod-cidr="" Jul 13 17:05:02.569846 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250533 2569 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 13 17:05:02.569846 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250539 2569 flags.go:64] FLAG: --pod-manifest-path="" Jul 13 17:05:02.569846 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250542 2569 flags.go:64] FLAG: --pod-max-pids="-1" Jul 13 17:05:02.569846 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250545 2569 flags.go:64] FLAG: --pods-per-core="0" Jul 13 17:05:02.569846 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250548 2569 flags.go:64] FLAG: --port="10250" Jul 13 17:05:02.569846 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250551 2569 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 13 17:05:02.569846 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250554 2569 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-086e4afe735a8d814" Jul 13 17:05:02.569846 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250557 2569 flags.go:64] FLAG: --qos-reserved="" Jul 13 17:05:02.569846 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250560 2569 flags.go:64] FLAG: --read-only-port="10255" Jul 13 17:05:02.569846 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250563 2569 flags.go:64] FLAG: --register-node="true" Jul 13 17:05:02.569846 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250566 2569 flags.go:64] FLAG: --register-with-taints="" Jul 13 17:05:02.569846 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250570 2569 flags.go:64] FLAG: --registry-burst="10" Jul 13 17:05:02.569846 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250573 2569 flags.go:64] FLAG: --registry-qps="5" Jul 13 17:05:02.569846 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250576 2569 flags.go:64] FLAG: --reserved-cpus="" Jul 13 17:05:02.569846 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250578 2569 flags.go:64] FLAG: --reserved-memory="" Jul 13 17:05:02.569846 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250582 2569 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 13 17:05:02.569846 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250586 2569 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 13 17:05:02.569846 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250590 2569 flags.go:64] FLAG: --rotate-certificates="false" Jul 13 17:05:02.569846 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250592 2569 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 13 17:05:02.569846 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250595 2569 flags.go:64] FLAG: --runonce="false" Jul 13 17:05:02.569846 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250598 2569 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 13 17:05:02.569846 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250602 2569 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 13 17:05:02.569846 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250605 2569 flags.go:64] FLAG: --seccomp-default="false" Jul 13 17:05:02.569846 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250607 2569 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 13 17:05:02.569846 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250610 2569 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 13 17:05:02.743989 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250613 2569 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 13 17:05:02.743989 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250617 2569 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 13 17:05:02.743989 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250623 2569 flags.go:64] FLAG: --storage-driver-password="root" Jul 13 17:05:02.743989 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250626 2569 flags.go:64] FLAG: --storage-driver-secure="false" Jul 13 17:05:02.743989 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250628 2569 flags.go:64] FLAG: --storage-driver-table="stats" Jul 13 17:05:02.743989 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250631 2569 flags.go:64] FLAG: --storage-driver-user="root" Jul 13 17:05:02.743989 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250634 2569 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 13 17:05:02.743989 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250637 2569 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 13 17:05:02.743989 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250640 2569 flags.go:64] FLAG: --system-cgroups="" Jul 13 17:05:02.743989 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250643 2569 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 13 17:05:02.743989 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250648 2569 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 13 17:05:02.743989 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250651 2569 flags.go:64] FLAG: --tls-cert-file="" Jul 13 17:05:02.743989 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250654 2569 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 13 17:05:02.743989 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250658 2569 flags.go:64] FLAG: --tls-min-version="" Jul 13 17:05:02.743989 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250660 2569 flags.go:64] FLAG: --tls-private-key-file="" Jul 13 17:05:02.743989 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250663 2569 flags.go:64] FLAG: --topology-manager-policy="none" Jul 13 17:05:02.743989 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250666 2569 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 13 17:05:02.743989 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250669 2569 flags.go:64] FLAG: --topology-manager-scope="container" Jul 13 17:05:02.743989 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250672 2569 flags.go:64] FLAG: --v="2" Jul 13 17:05:02.743989 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250676 2569 flags.go:64] FLAG: --version="false" Jul 13 17:05:02.743989 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250680 2569 flags.go:64] FLAG: --vmodule="" Jul 13 17:05:02.743989 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250683 2569 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 13 17:05:02.743989 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.250686 2569 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 13 17:05:02.743989 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251242 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 17:05:02.743989 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251261 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 17:05:02.744780 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251266 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 17:05:02.744780 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251271 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 17:05:02.744780 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251276 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 17:05:02.744780 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251280 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 17:05:02.744780 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251285 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 17:05:02.744780 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251289 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 17:05:02.744780 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251294 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 17:05:02.744780 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251298 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 17:05:02.744780 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251303 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 17:05:02.744780 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251313 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 17:05:02.744780 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251317 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 17:05:02.744780 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251323 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 17:05:02.744780 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251328 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 17:05:02.744780 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251332 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 17:05:02.744780 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251337 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 17:05:02.744780 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251342 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 17:05:02.744780 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251347 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 17:05:02.744780 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251352 2569 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 17:05:02.744780 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251356 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 17:05:02.744780 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251360 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 17:05:02.745323 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251364 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 17:05:02.745323 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251369 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 17:05:02.745323 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251378 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 17:05:02.745323 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251383 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 17:05:02.745323 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251388 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 17:05:02.745323 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251392 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 17:05:02.745323 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251397 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 17:05:02.745323 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251401 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 17:05:02.745323 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251405 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 17:05:02.745323 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251410 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 17:05:02.745323 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251417 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 17:05:02.745323 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251425 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 17:05:02.745323 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251430 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 17:05:02.745323 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251434 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 17:05:02.745323 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251443 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 17:05:02.745323 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251447 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 17:05:02.745323 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251451 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 17:05:02.745323 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251456 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 17:05:02.745323 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251460 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 17:05:02.745861 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251464 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 17:05:02.745861 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251468 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 17:05:02.745861 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251473 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 17:05:02.745861 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251477 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 17:05:02.745861 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251481 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 17:05:02.745861 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251490 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 17:05:02.745861 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251494 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 17:05:02.745861 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251498 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 17:05:02.745861 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251507 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 17:05:02.745861 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251511 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 17:05:02.745861 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251515 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 17:05:02.745861 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251520 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 17:05:02.745861 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251524 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 17:05:02.745861 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251530 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 17:05:02.745861 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251536 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 17:05:02.745861 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251540 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 17:05:02.745861 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251548 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 17:05:02.745861 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251553 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 17:05:02.745861 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251558 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 17:05:02.746384 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251563 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 17:05:02.746384 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251572 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 17:05:02.746384 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251577 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 17:05:02.746384 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251582 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 17:05:02.746384 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251587 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 17:05:02.746384 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251627 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 17:05:02.746384 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251654 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 17:05:02.746384 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251686 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 17:05:02.746384 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251790 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 17:05:02.746384 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251795 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 17:05:02.746384 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251802 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 17:05:02.746384 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251807 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 17:05:02.746384 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251810 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 17:05:02.746384 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251813 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 17:05:02.746384 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251817 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 17:05:02.746384 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251820 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 17:05:02.746384 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251823 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 17:05:02.746384 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251826 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 17:05:02.746384 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251829 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 17:05:02.746384 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251832 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 17:05:02.746933 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251835 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 17:05:02.746933 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251838 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 17:05:02.746933 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251841 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 17:05:02.746933 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251844 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 17:05:02.746933 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251846 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 17:05:02.746933 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251849 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 17:05:02.746933 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251852 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 17:05:02.746933 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251855 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 17:05:02.746933 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251858 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 17:05:02.746933 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251861 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 17:05:02.746933 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251864 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 17:05:02.746933 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251867 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 17:05:02.746933 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251869 2569 feature_gate.go:328] unrecognized feature gate: Example Jul 13 17:05:02.746933 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251872 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 17:05:02.746933 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251875 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 17:05:02.746933 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251878 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 17:05:02.746933 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251880 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 17:05:02.746933 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251883 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 17:05:02.746933 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.251886 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 17:05:02.747508 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.252691 2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 13 17:05:02.747508 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.260608 2569 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 13 17:05:02.747508 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.260622 2569 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 13 17:05:02.747508 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.261442 2569 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 13 17:05:02.747508 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.261450 2569 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 13 17:05:02.747508 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261510 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 17:05:02.747508 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261515 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 17:05:02.747508 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261519 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 17:05:02.747508 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261521 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 17:05:02.747508 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261526 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 17:05:02.747508 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261530 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 17:05:02.747508 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261533 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 17:05:02.747508 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261537 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 17:05:02.747508 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261540 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 17:05:02.747508 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261543 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 17:05:02.747956 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261546 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 17:05:02.747956 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261549 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 17:05:02.747956 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261552 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 17:05:02.747956 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261556 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 17:05:02.747956 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261559 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 17:05:02.747956 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261562 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 17:05:02.747956 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261564 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 17:05:02.747956 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261567 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 17:05:02.747956 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261572 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 17:05:02.747956 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261575 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 17:05:02.747956 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261581 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 17:05:02.747956 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261584 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 17:05:02.747956 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261587 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 17:05:02.747956 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261590 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 17:05:02.747956 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261593 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 17:05:02.747956 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261597 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 17:05:02.747956 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261600 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 17:05:02.747956 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261603 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 17:05:02.747956 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261606 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 17:05:02.748480 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261609 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 17:05:02.748480 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261612 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 17:05:02.748480 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261615 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 17:05:02.748480 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261618 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 17:05:02.748480 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261621 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 17:05:02.748480 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261624 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 17:05:02.748480 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261627 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 17:05:02.748480 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261630 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 17:05:02.748480 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261633 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 17:05:02.748480 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261635 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 17:05:02.748480 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261638 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 17:05:02.748480 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261640 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 17:05:02.748480 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261643 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 17:05:02.748480 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261645 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 17:05:02.748480 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261648 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 17:05:02.748480 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261651 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 17:05:02.748480 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261654 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 17:05:02.748480 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261656 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 17:05:02.748480 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261659 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 17:05:02.748480 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261662 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 17:05:02.749034 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261664 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 17:05:02.749034 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261668 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 17:05:02.749034 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261670 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 17:05:02.749034 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261674 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 17:05:02.749034 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261677 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 17:05:02.749034 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261679 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 17:05:02.749034 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261682 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 17:05:02.749034 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261685 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 17:05:02.749034 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261689 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 17:05:02.749034 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261691 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 17:05:02.749034 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261694 2569 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 17:05:02.749034 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261697 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 17:05:02.749034 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261700 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 17:05:02.749034 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261702 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 17:05:02.749034 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261705 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 17:05:02.749034 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261708 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 17:05:02.749034 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261710 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 17:05:02.749034 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261713 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 17:05:02.749034 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261715 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 17:05:02.749547 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261718 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 17:05:02.749547 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261721 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 17:05:02.749547 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261723 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 17:05:02.749547 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261726 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 17:05:02.749547 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261728 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 17:05:02.749547 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261731 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 17:05:02.749547 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261734 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 17:05:02.749547 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261736 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 17:05:02.749547 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261739 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 17:05:02.749547 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261748 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 17:05:02.749547 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261751 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 17:05:02.749547 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261753 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 17:05:02.749547 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261756 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 17:05:02.749547 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261758 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 17:05:02.749547 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261761 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 17:05:02.749547 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261764 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 17:05:02.749547 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261768 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 17:05:02.749547 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261770 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 17:05:02.749547 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261774 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 17:05:02.749547 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261778 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 17:05:02.750119 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261780 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 17:05:02.750119 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261784 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 17:05:02.750119 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261787 2569 feature_gate.go:328] unrecognized feature gate: Example Jul 13 17:05:02.750119 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261790 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 17:05:02.750119 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261792 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 17:05:02.750119 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261795 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 17:05:02.750119 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261798 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 17:05:02.750119 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261800 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 17:05:02.750119 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261803 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 17:05:02.750119 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261805 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 17:05:02.750119 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261808 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 17:05:02.750119 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.261813 2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 13 17:05:02.750119 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261936 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 17:05:02.750119 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261941 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 17:05:02.750119 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261944 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 17:05:02.750544 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261947 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 17:05:02.750544 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261951 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 17:05:02.750544 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261953 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 17:05:02.750544 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261956 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 17:05:02.750544 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261959 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 17:05:02.750544 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261961 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 17:05:02.750544 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261966 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 17:05:02.750544 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261973 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 17:05:02.750544 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261976 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 17:05:02.750544 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261979 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 17:05:02.750544 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261981 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 17:05:02.750544 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261984 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 17:05:02.750544 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261987 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 17:05:02.750544 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261991 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 17:05:02.750544 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261993 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 17:05:02.750544 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261996 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 17:05:02.750544 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.261998 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 17:05:02.750544 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262001 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 17:05:02.750544 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262003 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 17:05:02.750544 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262008 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 17:05:02.751127 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262010 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 17:05:02.751127 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262013 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 17:05:02.751127 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262016 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 17:05:02.751127 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262018 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 17:05:02.751127 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262021 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 17:05:02.751127 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262023 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 17:05:02.751127 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262026 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 17:05:02.751127 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262029 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 17:05:02.751127 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262031 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 17:05:02.751127 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262034 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 17:05:02.751127 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262036 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 17:05:02.751127 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262039 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 17:05:02.751127 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262042 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 17:05:02.751127 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262044 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 17:05:02.751127 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262047 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 17:05:02.751127 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262050 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 17:05:02.751127 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262052 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 17:05:02.751127 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262055 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 17:05:02.751127 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262058 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 17:05:02.751127 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262061 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 17:05:02.751647 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262069 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 17:05:02.751647 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262071 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 17:05:02.751647 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262074 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 17:05:02.751647 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262093 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 17:05:02.751647 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262096 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 17:05:02.751647 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262100 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 17:05:02.751647 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262103 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 17:05:02.751647 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262106 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 17:05:02.751647 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262109 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 17:05:02.751647 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262112 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 17:05:02.751647 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262114 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 17:05:02.751647 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262118 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 17:05:02.751647 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262121 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 17:05:02.751647 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262124 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 17:05:02.751647 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262127 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 17:05:02.751647 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262129 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 17:05:02.751647 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262132 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 17:05:02.751647 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262135 2569 feature_gate.go:328] unrecognized feature gate: Example Jul 13 17:05:02.751647 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262137 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 17:05:02.752232 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262140 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 17:05:02.752232 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262142 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 17:05:02.752232 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262145 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 17:05:02.752232 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262148 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 17:05:02.752232 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262150 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 17:05:02.752232 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262153 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 17:05:02.752232 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262156 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 17:05:02.752232 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262158 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 17:05:02.752232 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262161 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 17:05:02.752232 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262164 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 17:05:02.752232 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262168 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 17:05:02.752232 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262171 2569 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 17:05:02.752232 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262174 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 17:05:02.752232 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262177 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 17:05:02.752232 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262181 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 17:05:02.752232 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262184 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 17:05:02.752232 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262187 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 17:05:02.752232 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262190 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 17:05:02.752232 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262194 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 17:05:02.752734 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262197 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 17:05:02.752734 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262199 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 17:05:02.752734 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262202 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 17:05:02.752734 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262204 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 17:05:02.752734 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262207 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 17:05:02.752734 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262211 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 17:05:02.752734 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262213 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 17:05:02.752734 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262216 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 17:05:02.752734 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262219 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 17:05:02.752734 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262222 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 17:05:02.752734 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262226 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 17:05:02.752734 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262229 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 17:05:02.752734 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262232 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 17:05:02.752734 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262235 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 17:05:02.752734 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262238 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 17:05:02.752734 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262240 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 17:05:02.752734 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262243 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 17:05:02.752734 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:02.262245 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 17:05:02.753259 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.262250 2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 13 17:05:02.753259 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.262352 2569 server.go:962] "Client rotation is on, will bootstrap in background" Jul 13 17:05:02.753259 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.265323 2569 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 13 17:05:02.753259 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.266774 2569 server.go:1019] "Starting client certificate rotation" Jul 13 17:05:02.753259 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.266878 2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 13 17:05:02.753259 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.266913 2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 13 17:05:02.753259 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.269470 2569 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 13 17:05:02.753259 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.271797 2569 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 13 17:05:02.753259 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.285857 2569 log.go:25] "Validated CRI v1 runtime API" Jul 13 17:05:02.753259 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.289955 2569 log.go:25] "Validated CRI v1 image API" Jul 13 17:05:02.753259 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.290731 2569 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 13 17:05:02.753259 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.295473 2569 fs.go:135] Filesystem UUIDs: map[162aa95f-63a4-4161-8f08-dd48bc09686e:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 b293b4bd-fe2c-4221-bb1a-93d70599247f:/dev/nvme0n1p4] Jul 13 17:05:02.753604 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.295492 2569 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 13 17:05:02.753604 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.298534 2569 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 13 17:05:02.754042 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.300563 2569 manager.go:216] Machine: {Timestamp:2026-07-13 17:05:02.299204278 +0000 UTC m=+0.383348129 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101255 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2c6a20aa5daf82506099744a9ff471 SystemUUID:ec2c6a20-aa5d-af82-5060-99744a9ff471 BootID:389e4e37-da41-4b17-99d9-1102d99a3d0b Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:31:fc:3f:75:39 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:31:fc:3f:75:39 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:6a:36:5c:34:9b:30 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 13 17:05:02.754042 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.300659 2569 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 13 17:05:02.754042 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.300732 2569 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 13 17:05:02.754042 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.304790 2569 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 13 17:05:02.754042 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.304811 2569 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-220.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 13 17:05:02.754042 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.306516 2569 topology_manager.go:138] "Creating topology manager with none policy" Jul 13 17:05:02.754042 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.306526 2569 container_manager_linux.go:309] "Creating device plugin manager" Jul 13 17:05:02.754042 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.306534 2569 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 13 17:05:02.754042 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.306544 2569 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 13 17:05:02.754042 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.308144 2569 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 13 17:05:02.754042 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.311515 2569 state_mem.go:36] "Initialized new in-memory state store" Jul 13 17:05:02.754042 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.311630 2569 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 13 17:05:02.754042 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.311660 2569 kubelet.go:486] "Attempting to sync node with API server" Jul 13 17:05:02.754042 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.311670 2569 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 13 17:05:02.754042 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.311683 2569 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 13 17:05:02.754042 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.311693 2569 kubelet.go:398] "Adding apiserver pod source" Jul 13 17:05:02.754042 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.311702 2569 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 13 17:05:02.754042 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.311995 2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 13 17:05:02.951202 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.312009 2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 13 17:05:02.951202 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.314367 2569 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 13 17:05:02.951202 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.316939 2569 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 13 17:05:02.951202 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.316967 2569 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 13 17:05:02.951202 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.317124 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 13 17:05:02.951202 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.317136 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 13 17:05:02.951202 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.317142 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 13 17:05:02.951202 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.317147 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 13 17:05:02.951202 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.317154 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 13 17:05:02.951202 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.317159 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 13 17:05:02.951202 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.317837 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 13 17:05:02.951202 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.317844 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 13 17:05:02.951202 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.317851 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 13 17:05:02.951202 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.317857 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 13 17:05:02.951202 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.317865 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 13 17:05:02.951202 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.317873 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 13 17:05:02.951202 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.318734 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 13 17:05:02.951202 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.318743 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 13 17:05:02.951202 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.324177 2569 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-220.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 13 17:05:02.951202 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:02.324393 2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 13 17:05:02.951202 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:02.324439 2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-220.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 13 17:05:02.951202 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.324775 2569 server.go:1268] "Started kubelet" Jul 13 17:05:02.951202 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.324885 2569 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 13 17:05:02.951202 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.324872 2569 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 13 17:05:02.951202 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.324939 2569 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 13 17:05:02.951202 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.326307 2569 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 13 17:05:02.951202 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.327508 2569 server.go:310] "Adding debug handlers to kubelet server" Jul 13 17:05:02.951202 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:02.329754 2569 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 13 17:05:02.951202 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:02.331101 2569 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-220.ec2.internal.18c1e8431abe9ce4 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-220.ec2.internal,UID:ip-10-0-136-220.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-220.ec2.internal,},FirstTimestamp:2026-07-13 17:05:02.324579556 +0000 UTC m=+0.408723407,LastTimestamp:2026-07-13 17:05:02.324579556 +0000 UTC m=+0.408723407,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-220.ec2.internal,}" Jul 13 17:05:02.951202 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.335258 2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 13 17:05:02.951202 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.335258 2569 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 13 17:05:02.951202 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:02.335507 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-220.ec2.internal\" not found" Jul 13 17:05:02.951202 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.336121 2569 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 13 17:05:02.951202 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.336158 2569 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 13 17:05:02.951202 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.336325 2569 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 13 17:05:02.951202 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.336403 2569 reconstruct.go:97] "Volume reconstruction finished" Jul 13 17:05:02.951202 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.336412 2569 reconciler.go:29] "Reconciler: start to sync state" Jul 13 17:05:02.951202 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.337254 2569 factory.go:55] Registering systemd factory Jul 13 17:05:02.951202 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.337302 2569 factory.go:223] Registration of the systemd container factory successfully Jul 13 17:05:02.951202 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.337897 2569 factory.go:153] Registering CRI-O factory Jul 13 17:05:02.951202 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.337916 2569 factory.go:223] Registration of the crio container factory successfully Jul 13 17:05:02.954312 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.338092 2569 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 13 17:05:02.954312 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:02.338316 2569 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 13 17:05:02.954312 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.338339 2569 factory.go:103] Registering Raw factory Jul 13 17:05:02.954312 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.338402 2569 manager.go:1196] Started watching for new ooms in manager Jul 13 17:05:02.954312 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:02.338427 2569 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-220.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 13 17:05:02.954312 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.339169 2569 manager.go:349] Starting recovery of all containers Jul 13 17:05:02.954312 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.345610 2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-t4rm6" Jul 13 17:05:02.954312 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.349928 2569 manager.go:354] Recovery completed Jul 13 17:05:02.954312 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.354622 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 17:05:02.954312 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.355968 2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-t4rm6" Jul 13 17:05:02.954312 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.356991 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-220.ec2.internal" event="NodeHasSufficientMemory" Jul 13 17:05:02.954312 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.357018 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-220.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 17:05:02.954312 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.357033 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-220.ec2.internal" event="NodeHasSufficientPID" Jul 13 17:05:02.954312 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.357746 2569 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 13 17:05:02.954312 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.357758 2569 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 13 17:05:02.954312 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.357753 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 17:05:02.954312 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.357776 2569 state_mem.go:36] "Initialized new in-memory state store" Jul 13 17:05:02.954312 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.358556 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-220.ec2.internal" event="NodeHasSufficientMemory" Jul 13 17:05:02.954312 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.358582 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-220.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 17:05:02.954312 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.358591 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-220.ec2.internal" event="NodeHasSufficientPID" Jul 13 17:05:02.954312 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.360091 2569 policy_none.go:49] "None policy: Start" Jul 13 17:05:02.954312 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.360104 2569 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 13 17:05:02.954312 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.360115 2569 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 13 17:05:02.954312 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.361555 2569 policy_none.go:47] "Start" Jul 13 17:05:02.954312 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:02.436022 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-220.ec2.internal\" not found" Jul 13 17:05:02.954312 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.438741 2569 manager.go:337] "Starting Device Plugin manager" Jul 13 17:05:02.954312 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:02.438772 2569 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 13 17:05:02.954312 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.438786 2569 server.go:87] "Starting device plugin registration server" Jul 13 17:05:02.954312 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.439039 2569 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 13 17:05:02.954312 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.439052 2569 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 13 17:05:02.954312 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.439123 2569 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 13 17:05:02.954312 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.439213 2569 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 13 17:05:02.954312 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.439221 2569 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 13 17:05:02.954312 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:02.440156 2569 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 13 17:05:02.954312 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:02.440222 2569 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-220.ec2.internal\" not found" Jul 13 17:05:02.954312 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.442728 2569 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 13 17:05:02.954312 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.443931 2569 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 13 17:05:02.954312 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.443964 2569 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 13 17:05:02.954312 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.443982 2569 kubelet.go:2448] "Starting kubelet main sync loop" Jul 13 17:05:02.954312 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:02.444024 2569 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 13 17:05:02.954312 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.446388 2569 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 17:05:02.954312 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.539820 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 17:05:02.954312 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.540643 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-220.ec2.internal" event="NodeHasSufficientMemory" Jul 13 17:05:02.954312 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.540672 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-220.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 17:05:02.954312 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.540682 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-220.ec2.internal" event="NodeHasSufficientPID" Jul 13 17:05:02.955542 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.540703 2569 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-220.ec2.internal" Jul 13 17:05:02.955542 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.544247 2569 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-220.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-220.ec2.internal"] Jul 13 17:05:02.955542 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:02.544291 2569 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-136-220.ec2.internal\" not found" node="ip-10-0-136-220.ec2.internal" Jul 13 17:05:02.955542 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.544329 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 17:05:02.955542 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.545705 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-220.ec2.internal" event="NodeHasSufficientMemory" Jul 13 17:05:02.955542 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.545726 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-220.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 17:05:02.955542 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.545739 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-220.ec2.internal" event="NodeHasSufficientPID" Jul 13 17:05:02.955542 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.546964 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 17:05:02.955542 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.547124 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-220.ec2.internal" Jul 13 17:05:02.955542 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.547160 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 17:05:02.955542 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.547875 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-220.ec2.internal" event="NodeHasSufficientMemory" Jul 13 17:05:02.955542 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.547900 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-220.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 17:05:02.955542 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.547909 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-220.ec2.internal" event="NodeHasSufficientPID" Jul 13 17:05:02.955542 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.547919 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-220.ec2.internal" event="NodeHasSufficientMemory" Jul 13 17:05:02.955542 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.547941 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-220.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 17:05:02.955542 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.547954 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-220.ec2.internal" event="NodeHasSufficientPID" Jul 13 17:05:02.955542 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.549417 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-220.ec2.internal" Jul 13 17:05:02.955542 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.549438 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 17:05:02.955542 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.550071 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-220.ec2.internal" event="NodeHasSufficientMemory" Jul 13 17:05:02.955542 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.550108 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-220.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 17:05:02.955542 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.550120 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-220.ec2.internal" event="NodeHasSufficientPID" Jul 13 17:05:02.955542 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.557507 2569 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-220.ec2.internal" Jul 13 17:05:02.955542 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:02.557538 2569 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-220.ec2.internal\": node \"ip-10-0-136-220.ec2.internal\" not found" Jul 13 17:05:02.955542 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:02.582515 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-220.ec2.internal\" not found" Jul 13 17:05:02.955542 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:02.586770 2569 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-220.ec2.internal\" not found" node="ip-10-0-136-220.ec2.internal" Jul 13 17:05:02.955542 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:02.599539 2569 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-220.ec2.internal\" not found" node="ip-10-0-136-220.ec2.internal" Jul 13 17:05:02.955542 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.638608 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d00899e62aada42f2f0bdc458167583d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-220.ec2.internal\" (UID: \"d00899e62aada42f2f0bdc458167583d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-220.ec2.internal" Jul 13 17:05:02.955542 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.638632 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/45606745bd8d4c83782613a3e901378b-config\") pod \"kube-apiserver-proxy-ip-10-0-136-220.ec2.internal\" (UID: \"45606745bd8d4c83782613a3e901378b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-220.ec2.internal" Jul 13 17:05:02.955542 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.638650 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d00899e62aada42f2f0bdc458167583d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-220.ec2.internal\" (UID: \"d00899e62aada42f2f0bdc458167583d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-220.ec2.internal" Jul 13 17:05:02.955542 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:02.682553 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-220.ec2.internal\" not found" Jul 13 17:05:02.955542 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.739399 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d00899e62aada42f2f0bdc458167583d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-220.ec2.internal\" (UID: \"d00899e62aada42f2f0bdc458167583d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-220.ec2.internal" Jul 13 17:05:02.956409 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.739422 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/45606745bd8d4c83782613a3e901378b-config\") pod \"kube-apiserver-proxy-ip-10-0-136-220.ec2.internal\" (UID: \"45606745bd8d4c83782613a3e901378b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-220.ec2.internal" Jul 13 17:05:02.956409 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.739476 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d00899e62aada42f2f0bdc458167583d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-220.ec2.internal\" (UID: \"d00899e62aada42f2f0bdc458167583d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-220.ec2.internal" Jul 13 17:05:02.956409 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.739514 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d00899e62aada42f2f0bdc458167583d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-220.ec2.internal\" (UID: \"d00899e62aada42f2f0bdc458167583d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-220.ec2.internal" Jul 13 17:05:02.956409 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.739538 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/45606745bd8d4c83782613a3e901378b-config\") pod \"kube-apiserver-proxy-ip-10-0-136-220.ec2.internal\" (UID: \"45606745bd8d4c83782613a3e901378b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-220.ec2.internal" Jul 13 17:05:02.956409 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.739436 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d00899e62aada42f2f0bdc458167583d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-220.ec2.internal\" (UID: \"d00899e62aada42f2f0bdc458167583d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-220.ec2.internal" Jul 13 17:05:02.956409 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:02.783603 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-220.ec2.internal\" not found" Jul 13 17:05:02.956409 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:02.884165 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-220.ec2.internal\" not found" Jul 13 17:05:02.956409 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.888317 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-220.ec2.internal" Jul 13 17:05:02.956409 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:02.902832 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-220.ec2.internal" Jul 13 17:05:03.171893 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:02.984462 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-220.ec2.internal\" not found" Jul 13 17:05:03.171893 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:03.084911 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-220.ec2.internal\" not found" Jul 13 17:05:03.385591 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:03.185366 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-220.ec2.internal\" not found" Jul 13 17:05:03.385591 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:03.265857 2569 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 13 17:05:03.385591 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:03.266005 2569 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 13 17:05:03.385591 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:03.286125 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-220.ec2.internal\" not found" Jul 13 17:05:03.385591 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:03.361327 2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-12 17:00:04 +0000 UTC" deadline="2028-01-28 21:57:31.60094441 +0000 UTC" Jul 13 17:05:03.385591 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:03.361361 2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13540h52m28.239586231s" Jul 13 17:05:03.386648 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:03.386483 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-220.ec2.internal\" not found" Jul 13 17:05:05.266886 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:05.164068 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd00899e62aada42f2f0bdc458167583d.slice/crio-ae76710ebbfbe227d2d77a7f16f87c70c644ebf1c2f7ee8621e021d30a432607 WatchSource:0}: Error finding container ae76710ebbfbe227d2d77a7f16f87c70c644ebf1c2f7ee8621e021d30a432607: Status 404 returned error can't find the container with id ae76710ebbfbe227d2d77a7f16f87c70c644ebf1c2f7ee8621e021d30a432607 Jul 13 17:05:05.266886 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:05.167482 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45606745bd8d4c83782613a3e901378b.slice/crio-58c6a41a521cef6294ca68dacf1555ad293426f5d4640b795404fe690da28dba WatchSource:0}: Error finding container 58c6a41a521cef6294ca68dacf1555ad293426f5d4640b795404fe690da28dba: Status 404 returned error can't find the container with id 58c6a41a521cef6294ca68dacf1555ad293426f5d4640b795404fe690da28dba Jul 13 17:05:05.266886 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:05.169372 2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 17:05:05.266886 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:05.177714 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-220.ec2.internal\" not found" Jul 13 17:05:05.266886 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:05.212263 2569 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 17:05:05.462016 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:05.277989 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-220.ec2.internal\" not found" Jul 13 17:05:05.462016 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:05.378553 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-220.ec2.internal\" not found" Jul 13 17:05:05.645528 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:05.467524 2569 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 17:05:05.645528 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:05.478992 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-220.ec2.internal\" not found" Jul 13 17:05:05.645528 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:05.579603 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-220.ec2.internal\" not found" Jul 13 17:05:05.645528 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:05.607747 2569 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 17:05:05.645528 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:05.623382 2569 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-220.ec2.internal" Jul 13 17:05:05.645528 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:05.637413 2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 13 17:05:05.645528 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:05.638404 2569 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-220.ec2.internal" Jul 13 17:05:05.645528 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:05.645105 2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 13 17:05:05.821489 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:05.808932 2569 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 17:05:06.013438 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:05.999458 2569 apiserver.go:52] "Watching apiserver" Jul 13 17:05:06.013438 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.005056 2569 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 13 17:05:06.013438 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.006181 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-nfjpb","openshift-dns/node-resolver-qn7kc","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-220.ec2.internal","openshift-multus/multus-additional-cni-plugins-xpws4","openshift-multus/network-metrics-daemon-t6r98","openshift-network-diagnostics/network-check-target-2b6wx","openshift-network-operator/iptables-alerter-tnc2p","openshift-ovn-kubernetes/ovnkube-node-q2pnb","kube-system/konnectivity-agent-phcxr","kube-system/kube-apiserver-proxy-ip-10-0-136-220.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7g2s","openshift-cluster-node-tuning-operator/tuned-fhjzk","openshift-image-registry/node-ca-psn5f","openshift-multus/multus-cwgjn"] Jul 13 17:05:06.013438 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.007601 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tnc2p" Jul 13 17:05:06.013438 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.008906 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-psn5f" Jul 13 17:05:06.013438 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.009764 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-cjs7z\"" Jul 13 17:05:06.013438 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.009979 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 13 17:05:06.013438 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.010239 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 13 17:05:06.013438 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.010496 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 13 17:05:06.013438 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.010763 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-cwgjn" Jul 13 17:05:06.013438 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.010766 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 13 17:05:06.013438 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.011593 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 13 17:05:06.013438 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.011787 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-72vs6\"" Jul 13 17:05:06.013438 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.011940 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 13 17:05:06.013438 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.012291 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xpws4" Jul 13 17:05:06.013438 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.012714 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6r98" Jul 13 17:05:06.013438 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:06.012758 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6r98" podUID="7cb8e14a-393f-43f2-b75a-7f14756cfb2a" Jul 13 17:05:06.016698 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.016579 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 13 17:05:06.016949 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.016799 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-9nfcs\"" Jul 13 17:05:06.017503 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.017202 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 13 17:05:06.017503 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.017471 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 13 17:05:06.132212 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.018474 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-kqcv6\"" Jul 13 17:05:06.132212 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.018799 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 13 17:05:06.132212 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.018950 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2b6wx" Jul 13 17:05:06.132212 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:06.019023 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2b6wx" podUID="5d1dd68a-166a-4be1-812e-3e15e056c3f2" Jul 13 17:05:06.132212 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.019128 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 13 17:05:06.132212 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.019511 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 13 17:05:06.132212 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.019577 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-phcxr" Jul 13 17:05:06.132212 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.020670 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nfjpb" Jul 13 17:05:06.132212 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:06.020829 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nfjpb" podUID="2263bf98-5f88-41d9-b755-9f874a3d5916" Jul 13 17:05:06.132212 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.022643 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-qn7kc" Jul 13 17:05:06.132212 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.023749 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 13 17:05:06.132212 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.023794 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-cgb22\"" Jul 13 17:05:06.132212 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.024003 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 13 17:05:06.132212 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.024494 2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 13 17:05:06.132212 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.025145 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-7nfkq\"" Jul 13 17:05:06.132212 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.025180 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 13 17:05:06.132212 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.025387 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 13 17:05:06.132212 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.025545 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.132212 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.026816 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7g2s" Jul 13 17:05:06.132212 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.027255 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 13 17:05:06.132212 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.027431 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-fnbwf\"" Jul 13 17:05:06.132212 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.027479 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 13 17:05:06.132212 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.027600 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 13 17:05:06.132212 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.027633 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 13 17:05:06.132212 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.027709 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 13 17:05:06.132212 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.027725 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 13 17:05:06.132212 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.028142 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fhjzk" Jul 13 17:05:06.132212 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.028615 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 13 17:05:06.132212 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.028650 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-blvts\"" Jul 13 17:05:06.132212 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.028686 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 13 17:05:06.132212 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.028735 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 13 17:05:06.132212 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.030078 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 13 17:05:06.132212 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.030128 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 13 17:05:06.132212 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.030132 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-gcb9k\"" Jul 13 17:05:06.132212 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.041008 2569 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 13 17:05:06.134797 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.043054 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/252eeac8-ef53-439f-8d2f-8ef03a102104-host-run-multus-certs\") pod \"multus-cwgjn\" (UID: \"252eeac8-ef53-439f-8d2f-8ef03a102104\") " pod="openshift-multus/multus-cwgjn" Jul 13 17:05:06.134797 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.043083 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6a531b70-4a58-4752-8fa8-5ba634b542ea-var-lib-openvswitch\") pod \"ovnkube-node-q2pnb\" (UID: \"6a531b70-4a58-4752-8fa8-5ba634b542ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.134797 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.043100 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6a531b70-4a58-4752-8fa8-5ba634b542ea-ovnkube-config\") pod \"ovnkube-node-q2pnb\" (UID: \"6a531b70-4a58-4752-8fa8-5ba634b542ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.134797 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.043117 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75wpl\" (UniqueName: \"kubernetes.io/projected/6a531b70-4a58-4752-8fa8-5ba634b542ea-kube-api-access-75wpl\") pod \"ovnkube-node-q2pnb\" (UID: \"6a531b70-4a58-4752-8fa8-5ba634b542ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.134797 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.043145 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/252eeac8-ef53-439f-8d2f-8ef03a102104-multus-cni-dir\") pod \"multus-cwgjn\" (UID: \"252eeac8-ef53-439f-8d2f-8ef03a102104\") " pod="openshift-multus/multus-cwgjn" Jul 13 17:05:06.134797 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.043169 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2263bf98-5f88-41d9-b755-9f874a3d5916-original-pull-secret\") pod \"global-pull-secret-syncer-nfjpb\" (UID: \"2263bf98-5f88-41d9-b755-9f874a3d5916\") " pod="kube-system/global-pull-secret-syncer-nfjpb" Jul 13 17:05:06.134797 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.043278 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6hpn\" (UniqueName: \"kubernetes.io/projected/5d1dd68a-166a-4be1-812e-3e15e056c3f2-kube-api-access-z6hpn\") pod \"network-check-target-2b6wx\" (UID: \"5d1dd68a-166a-4be1-812e-3e15e056c3f2\") " pod="openshift-network-diagnostics/network-check-target-2b6wx" Jul 13 17:05:06.134797 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.043334 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/252eeac8-ef53-439f-8d2f-8ef03a102104-host-run-netns\") pod \"multus-cwgjn\" (UID: \"252eeac8-ef53-439f-8d2f-8ef03a102104\") " pod="openshift-multus/multus-cwgjn" Jul 13 17:05:06.134797 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.043385 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b3ec2a99-eadb-4865-ae2a-dd77aab461de-agent-certs\") pod \"konnectivity-agent-phcxr\" (UID: \"b3ec2a99-eadb-4865-ae2a-dd77aab461de\") " pod="kube-system/konnectivity-agent-phcxr" Jul 13 17:05:06.134797 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.043419 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b3ec2a99-eadb-4865-ae2a-dd77aab461de-konnectivity-ca\") pod \"konnectivity-agent-phcxr\" (UID: \"b3ec2a99-eadb-4865-ae2a-dd77aab461de\") " pod="kube-system/konnectivity-agent-phcxr" Jul 13 17:05:06.134797 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.043447 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6a531b70-4a58-4752-8fa8-5ba634b542ea-run-ovn\") pod \"ovnkube-node-q2pnb\" (UID: \"6a531b70-4a58-4752-8fa8-5ba634b542ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.134797 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.043466 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d401c2d6-40c6-4728-a35c-1efd3f17bb9f-host-slash\") pod \"iptables-alerter-tnc2p\" (UID: \"d401c2d6-40c6-4728-a35c-1efd3f17bb9f\") " pod="openshift-network-operator/iptables-alerter-tnc2p" Jul 13 17:05:06.134797 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.043500 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/921a5cfd-83c7-4db7-be13-53e44c5df655-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xpws4\" (UID: \"921a5cfd-83c7-4db7-be13-53e44c5df655\") " pod="openshift-multus/multus-additional-cni-plugins-xpws4" Jul 13 17:05:06.134797 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.043518 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2qvc\" (UniqueName: \"kubernetes.io/projected/a7a4b481-50e3-4bb3-bcb5-6c5942814726-kube-api-access-z2qvc\") pod \"node-resolver-qn7kc\" (UID: \"a7a4b481-50e3-4bb3-bcb5-6c5942814726\") " pod="openshift-dns/node-resolver-qn7kc" Jul 13 17:05:06.134797 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.043600 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/252eeac8-ef53-439f-8d2f-8ef03a102104-multus-socket-dir-parent\") pod \"multus-cwgjn\" (UID: \"252eeac8-ef53-439f-8d2f-8ef03a102104\") " pod="openshift-multus/multus-cwgjn" Jul 13 17:05:06.134797 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.043626 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jt287\" (UniqueName: \"kubernetes.io/projected/252eeac8-ef53-439f-8d2f-8ef03a102104-kube-api-access-jt287\") pod \"multus-cwgjn\" (UID: \"252eeac8-ef53-439f-8d2f-8ef03a102104\") " pod="openshift-multus/multus-cwgjn" Jul 13 17:05:06.136305 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.043649 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6a531b70-4a58-4752-8fa8-5ba634b542ea-systemd-units\") pod \"ovnkube-node-q2pnb\" (UID: \"6a531b70-4a58-4752-8fa8-5ba634b542ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.136305 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.043673 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6a531b70-4a58-4752-8fa8-5ba634b542ea-etc-openvswitch\") pod \"ovnkube-node-q2pnb\" (UID: \"6a531b70-4a58-4752-8fa8-5ba634b542ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.136305 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.043700 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/921a5cfd-83c7-4db7-be13-53e44c5df655-cni-binary-copy\") pod \"multus-additional-cni-plugins-xpws4\" (UID: \"921a5cfd-83c7-4db7-be13-53e44c5df655\") " pod="openshift-multus/multus-additional-cni-plugins-xpws4" Jul 13 17:05:06.136305 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.043731 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/921a5cfd-83c7-4db7-be13-53e44c5df655-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xpws4\" (UID: \"921a5cfd-83c7-4db7-be13-53e44c5df655\") " pod="openshift-multus/multus-additional-cni-plugins-xpws4" Jul 13 17:05:06.136305 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.043805 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/252eeac8-ef53-439f-8d2f-8ef03a102104-os-release\") pod \"multus-cwgjn\" (UID: \"252eeac8-ef53-439f-8d2f-8ef03a102104\") " pod="openshift-multus/multus-cwgjn" Jul 13 17:05:06.136305 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.043831 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/252eeac8-ef53-439f-8d2f-8ef03a102104-host-var-lib-cni-bin\") pod \"multus-cwgjn\" (UID: \"252eeac8-ef53-439f-8d2f-8ef03a102104\") " pod="openshift-multus/multus-cwgjn" Jul 13 17:05:06.136305 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.043855 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/252eeac8-ef53-439f-8d2f-8ef03a102104-hostroot\") pod \"multus-cwgjn\" (UID: \"252eeac8-ef53-439f-8d2f-8ef03a102104\") " pod="openshift-multus/multus-cwgjn" Jul 13 17:05:06.136305 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.043878 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/252eeac8-ef53-439f-8d2f-8ef03a102104-etc-kubernetes\") pod \"multus-cwgjn\" (UID: \"252eeac8-ef53-439f-8d2f-8ef03a102104\") " pod="openshift-multus/multus-cwgjn" Jul 13 17:05:06.136305 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.043911 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w27sn\" (UniqueName: \"kubernetes.io/projected/d401c2d6-40c6-4728-a35c-1efd3f17bb9f-kube-api-access-w27sn\") pod \"iptables-alerter-tnc2p\" (UID: \"d401c2d6-40c6-4728-a35c-1efd3f17bb9f\") " pod="openshift-network-operator/iptables-alerter-tnc2p" Jul 13 17:05:06.136305 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.043936 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/252eeac8-ef53-439f-8d2f-8ef03a102104-system-cni-dir\") pod \"multus-cwgjn\" (UID: \"252eeac8-ef53-439f-8d2f-8ef03a102104\") " pod="openshift-multus/multus-cwgjn" Jul 13 17:05:06.136305 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.044000 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6a531b70-4a58-4752-8fa8-5ba634b542ea-run-systemd\") pod \"ovnkube-node-q2pnb\" (UID: \"6a531b70-4a58-4752-8fa8-5ba634b542ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.136305 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.044054 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2263bf98-5f88-41d9-b755-9f874a3d5916-kubelet-config\") pod \"global-pull-secret-syncer-nfjpb\" (UID: \"2263bf98-5f88-41d9-b755-9f874a3d5916\") " pod="kube-system/global-pull-secret-syncer-nfjpb" Jul 13 17:05:06.136305 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.044097 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a7a4b481-50e3-4bb3-bcb5-6c5942814726-hosts-file\") pod \"node-resolver-qn7kc\" (UID: \"a7a4b481-50e3-4bb3-bcb5-6c5942814726\") " pod="openshift-dns/node-resolver-qn7kc" Jul 13 17:05:06.136305 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.044127 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/252eeac8-ef53-439f-8d2f-8ef03a102104-cni-binary-copy\") pod \"multus-cwgjn\" (UID: \"252eeac8-ef53-439f-8d2f-8ef03a102104\") " pod="openshift-multus/multus-cwgjn" Jul 13 17:05:06.136305 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.044151 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gx6wq\" (UniqueName: \"kubernetes.io/projected/7cb8e14a-393f-43f2-b75a-7f14756cfb2a-kube-api-access-gx6wq\") pod \"network-metrics-daemon-t6r98\" (UID: \"7cb8e14a-393f-43f2-b75a-7f14756cfb2a\") " pod="openshift-multus/network-metrics-daemon-t6r98" Jul 13 17:05:06.136305 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.044184 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6a531b70-4a58-4752-8fa8-5ba634b542ea-host-cni-bin\") pod \"ovnkube-node-q2pnb\" (UID: \"6a531b70-4a58-4752-8fa8-5ba634b542ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.137386 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.044231 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6a531b70-4a58-4752-8fa8-5ba634b542ea-env-overrides\") pod \"ovnkube-node-q2pnb\" (UID: \"6a531b70-4a58-4752-8fa8-5ba634b542ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.137386 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.044266 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/252eeac8-ef53-439f-8d2f-8ef03a102104-host-var-lib-kubelet\") pod \"multus-cwgjn\" (UID: \"252eeac8-ef53-439f-8d2f-8ef03a102104\") " pod="openshift-multus/multus-cwgjn" Jul 13 17:05:06.137386 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.044286 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a531b70-4a58-4752-8fa8-5ba634b542ea-host-kubelet\") pod \"ovnkube-node-q2pnb\" (UID: \"6a531b70-4a58-4752-8fa8-5ba634b542ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.137386 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.044307 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6a531b70-4a58-4752-8fa8-5ba634b542ea-log-socket\") pod \"ovnkube-node-q2pnb\" (UID: \"6a531b70-4a58-4752-8fa8-5ba634b542ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.137386 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.044330 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d401c2d6-40c6-4728-a35c-1efd3f17bb9f-iptables-alerter-script\") pod \"iptables-alerter-tnc2p\" (UID: \"d401c2d6-40c6-4728-a35c-1efd3f17bb9f\") " pod="openshift-network-operator/iptables-alerter-tnc2p" Jul 13 17:05:06.137386 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.044353 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/99b9c479-5323-44a6-bbfa-2aa301efb6af-host\") pod \"node-ca-psn5f\" (UID: \"99b9c479-5323-44a6-bbfa-2aa301efb6af\") " pod="openshift-image-registry/node-ca-psn5f" Jul 13 17:05:06.137386 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.044374 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/921a5cfd-83c7-4db7-be13-53e44c5df655-cnibin\") pod \"multus-additional-cni-plugins-xpws4\" (UID: \"921a5cfd-83c7-4db7-be13-53e44c5df655\") " pod="openshift-multus/multus-additional-cni-plugins-xpws4" Jul 13 17:05:06.137386 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.044395 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-869n2\" (UniqueName: \"kubernetes.io/projected/921a5cfd-83c7-4db7-be13-53e44c5df655-kube-api-access-869n2\") pod \"multus-additional-cni-plugins-xpws4\" (UID: \"921a5cfd-83c7-4db7-be13-53e44c5df655\") " pod="openshift-multus/multus-additional-cni-plugins-xpws4" Jul 13 17:05:06.137386 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.044419 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2263bf98-5f88-41d9-b755-9f874a3d5916-dbus\") pod \"global-pull-secret-syncer-nfjpb\" (UID: \"2263bf98-5f88-41d9-b755-9f874a3d5916\") " pod="kube-system/global-pull-secret-syncer-nfjpb" Jul 13 17:05:06.137386 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.044440 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a531b70-4a58-4752-8fa8-5ba634b542ea-host-run-ovn-kubernetes\") pod \"ovnkube-node-q2pnb\" (UID: \"6a531b70-4a58-4752-8fa8-5ba634b542ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.137386 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.044472 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6a531b70-4a58-4752-8fa8-5ba634b542ea-ovnkube-script-lib\") pod \"ovnkube-node-q2pnb\" (UID: \"6a531b70-4a58-4752-8fa8-5ba634b542ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.137386 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.044490 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/252eeac8-ef53-439f-8d2f-8ef03a102104-multus-conf-dir\") pod \"multus-cwgjn\" (UID: \"252eeac8-ef53-439f-8d2f-8ef03a102104\") " pod="openshift-multus/multus-cwgjn" Jul 13 17:05:06.137386 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.044503 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/252eeac8-ef53-439f-8d2f-8ef03a102104-multus-daemon-config\") pod \"multus-cwgjn\" (UID: \"252eeac8-ef53-439f-8d2f-8ef03a102104\") " pod="openshift-multus/multus-cwgjn" Jul 13 17:05:06.137386 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.044519 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a531b70-4a58-4752-8fa8-5ba634b542ea-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-q2pnb\" (UID: \"6a531b70-4a58-4752-8fa8-5ba634b542ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.137386 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.044541 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/252eeac8-ef53-439f-8d2f-8ef03a102104-host-var-lib-cni-multus\") pod \"multus-cwgjn\" (UID: \"252eeac8-ef53-439f-8d2f-8ef03a102104\") " pod="openshift-multus/multus-cwgjn" Jul 13 17:05:06.137386 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.044578 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7a4b481-50e3-4bb3-bcb5-6c5942814726-tmp-dir\") pod \"node-resolver-qn7kc\" (UID: \"a7a4b481-50e3-4bb3-bcb5-6c5942814726\") " pod="openshift-dns/node-resolver-qn7kc" Jul 13 17:05:06.138504 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.044600 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6a531b70-4a58-4752-8fa8-5ba634b542ea-host-cni-netd\") pod \"ovnkube-node-q2pnb\" (UID: \"6a531b70-4a58-4752-8fa8-5ba634b542ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.138504 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.044621 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/252eeac8-ef53-439f-8d2f-8ef03a102104-cnibin\") pod \"multus-cwgjn\" (UID: \"252eeac8-ef53-439f-8d2f-8ef03a102104\") " pod="openshift-multus/multus-cwgjn" Jul 13 17:05:06.138504 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.044643 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/252eeac8-ef53-439f-8d2f-8ef03a102104-host-run-k8s-cni-cncf-io\") pod \"multus-cwgjn\" (UID: \"252eeac8-ef53-439f-8d2f-8ef03a102104\") " pod="openshift-multus/multus-cwgjn" Jul 13 17:05:06.138504 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.044662 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7cb8e14a-393f-43f2-b75a-7f14756cfb2a-metrics-certs\") pod \"network-metrics-daemon-t6r98\" (UID: \"7cb8e14a-393f-43f2-b75a-7f14756cfb2a\") " pod="openshift-multus/network-metrics-daemon-t6r98" Jul 13 17:05:06.138504 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.044684 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6a531b70-4a58-4752-8fa8-5ba634b542ea-host-run-netns\") pod \"ovnkube-node-q2pnb\" (UID: \"6a531b70-4a58-4752-8fa8-5ba634b542ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.138504 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.044707 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6a531b70-4a58-4752-8fa8-5ba634b542ea-run-openvswitch\") pod \"ovnkube-node-q2pnb\" (UID: \"6a531b70-4a58-4752-8fa8-5ba634b542ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.138504 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.044725 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6a531b70-4a58-4752-8fa8-5ba634b542ea-node-log\") pod \"ovnkube-node-q2pnb\" (UID: \"6a531b70-4a58-4752-8fa8-5ba634b542ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.138504 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.044746 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6a531b70-4a58-4752-8fa8-5ba634b542ea-ovn-node-metrics-cert\") pod \"ovnkube-node-q2pnb\" (UID: \"6a531b70-4a58-4752-8fa8-5ba634b542ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.138504 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.044777 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/99b9c479-5323-44a6-bbfa-2aa301efb6af-serviceca\") pod \"node-ca-psn5f\" (UID: \"99b9c479-5323-44a6-bbfa-2aa301efb6af\") " pod="openshift-image-registry/node-ca-psn5f" Jul 13 17:05:06.138504 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.044791 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4w5m\" (UniqueName: \"kubernetes.io/projected/99b9c479-5323-44a6-bbfa-2aa301efb6af-kube-api-access-n4w5m\") pod \"node-ca-psn5f\" (UID: \"99b9c479-5323-44a6-bbfa-2aa301efb6af\") " pod="openshift-image-registry/node-ca-psn5f" Jul 13 17:05:06.138504 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.044806 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/921a5cfd-83c7-4db7-be13-53e44c5df655-system-cni-dir\") pod \"multus-additional-cni-plugins-xpws4\" (UID: \"921a5cfd-83c7-4db7-be13-53e44c5df655\") " pod="openshift-multus/multus-additional-cni-plugins-xpws4" Jul 13 17:05:06.138504 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.044833 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/921a5cfd-83c7-4db7-be13-53e44c5df655-os-release\") pod \"multus-additional-cni-plugins-xpws4\" (UID: \"921a5cfd-83c7-4db7-be13-53e44c5df655\") " pod="openshift-multus/multus-additional-cni-plugins-xpws4" Jul 13 17:05:06.138504 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.044866 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6a531b70-4a58-4752-8fa8-5ba634b542ea-host-slash\") pod \"ovnkube-node-q2pnb\" (UID: \"6a531b70-4a58-4752-8fa8-5ba634b542ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.138504 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.044910 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/921a5cfd-83c7-4db7-be13-53e44c5df655-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xpws4\" (UID: \"921a5cfd-83c7-4db7-be13-53e44c5df655\") " pod="openshift-multus/multus-additional-cni-plugins-xpws4" Jul 13 17:05:06.138504 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.065699 2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-9r944" Jul 13 17:05:06.138504 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.079434 2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-9r944" Jul 13 17:05:06.138504 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.123981 2569 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 13 17:05:06.254000 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.148408 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f4054b54-56fb-46d9-8302-2618330b5e9a-var-lib-kubelet\") pod \"tuned-fhjzk\" (UID: \"f4054b54-56fb-46d9-8302-2618330b5e9a\") " pod="openshift-cluster-node-tuning-operator/tuned-fhjzk" Jul 13 17:05:06.254000 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.148555 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/252eeac8-ef53-439f-8d2f-8ef03a102104-multus-cni-dir\") pod \"multus-cwgjn\" (UID: \"252eeac8-ef53-439f-8d2f-8ef03a102104\") " pod="openshift-multus/multus-cwgjn" Jul 13 17:05:06.254000 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.148602 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f4054b54-56fb-46d9-8302-2618330b5e9a-tmp\") pod \"tuned-fhjzk\" (UID: \"f4054b54-56fb-46d9-8302-2618330b5e9a\") " pod="openshift-cluster-node-tuning-operator/tuned-fhjzk" Jul 13 17:05:06.254000 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.148640 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2263bf98-5f88-41d9-b755-9f874a3d5916-original-pull-secret\") pod \"global-pull-secret-syncer-nfjpb\" (UID: \"2263bf98-5f88-41d9-b755-9f874a3d5916\") " pod="kube-system/global-pull-secret-syncer-nfjpb" Jul 13 17:05:06.254000 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:06.148879 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 17:05:06.254000 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.148852 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/252eeac8-ef53-439f-8d2f-8ef03a102104-multus-cni-dir\") pod \"multus-cwgjn\" (UID: \"252eeac8-ef53-439f-8d2f-8ef03a102104\") " pod="openshift-multus/multus-cwgjn" Jul 13 17:05:06.254000 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.149172 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f4054b54-56fb-46d9-8302-2618330b5e9a-etc-systemd\") pod \"tuned-fhjzk\" (UID: \"f4054b54-56fb-46d9-8302-2618330b5e9a\") " pod="openshift-cluster-node-tuning-operator/tuned-fhjzk" Jul 13 17:05:06.254000 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:06.153421 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2263bf98-5f88-41d9-b755-9f874a3d5916-original-pull-secret podName:2263bf98-5f88-41d9-b755-9f874a3d5916 nodeName:}" failed. No retries permitted until 2026-07-13 17:05:06.653386063 +0000 UTC m=+3.051044433 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2263bf98-5f88-41d9-b755-9f874a3d5916-original-pull-secret") pod "global-pull-secret-syncer-nfjpb" (UID: "2263bf98-5f88-41d9-b755-9f874a3d5916") : object "kube-system"/"original-pull-secret" not registered Jul 13 17:05:06.254000 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.153205 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxp6n\" (UniqueName: \"kubernetes.io/projected/f4054b54-56fb-46d9-8302-2618330b5e9a-kube-api-access-bxp6n\") pod \"tuned-fhjzk\" (UID: \"f4054b54-56fb-46d9-8302-2618330b5e9a\") " pod="openshift-cluster-node-tuning-operator/tuned-fhjzk" Jul 13 17:05:06.254000 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.153699 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6hpn\" (UniqueName: \"kubernetes.io/projected/5d1dd68a-166a-4be1-812e-3e15e056c3f2-kube-api-access-z6hpn\") pod \"network-check-target-2b6wx\" (UID: \"5d1dd68a-166a-4be1-812e-3e15e056c3f2\") " pod="openshift-network-diagnostics/network-check-target-2b6wx" Jul 13 17:05:06.254000 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.153811 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/252eeac8-ef53-439f-8d2f-8ef03a102104-host-run-netns\") pod \"multus-cwgjn\" (UID: \"252eeac8-ef53-439f-8d2f-8ef03a102104\") " pod="openshift-multus/multus-cwgjn" Jul 13 17:05:06.254000 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.153840 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b3ec2a99-eadb-4865-ae2a-dd77aab461de-agent-certs\") pod \"konnectivity-agent-phcxr\" (UID: \"b3ec2a99-eadb-4865-ae2a-dd77aab461de\") " pod="kube-system/konnectivity-agent-phcxr" Jul 13 17:05:06.254000 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.153872 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b3ec2a99-eadb-4865-ae2a-dd77aab461de-konnectivity-ca\") pod \"konnectivity-agent-phcxr\" (UID: \"b3ec2a99-eadb-4865-ae2a-dd77aab461de\") " pod="kube-system/konnectivity-agent-phcxr" Jul 13 17:05:06.254000 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.153902 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6a531b70-4a58-4752-8fa8-5ba634b542ea-run-ovn\") pod \"ovnkube-node-q2pnb\" (UID: \"6a531b70-4a58-4752-8fa8-5ba634b542ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.254000 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.153927 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/252eeac8-ef53-439f-8d2f-8ef03a102104-host-run-netns\") pod \"multus-cwgjn\" (UID: \"252eeac8-ef53-439f-8d2f-8ef03a102104\") " pod="openshift-multus/multus-cwgjn" Jul 13 17:05:06.254000 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.153932 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d401c2d6-40c6-4728-a35c-1efd3f17bb9f-host-slash\") pod \"iptables-alerter-tnc2p\" (UID: \"d401c2d6-40c6-4728-a35c-1efd3f17bb9f\") " pod="openshift-network-operator/iptables-alerter-tnc2p" Jul 13 17:05:06.254000 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.153984 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6a531b70-4a58-4752-8fa8-5ba634b542ea-run-ovn\") pod \"ovnkube-node-q2pnb\" (UID: \"6a531b70-4a58-4752-8fa8-5ba634b542ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.254625 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.154003 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/921a5cfd-83c7-4db7-be13-53e44c5df655-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xpws4\" (UID: \"921a5cfd-83c7-4db7-be13-53e44c5df655\") " pod="openshift-multus/multus-additional-cni-plugins-xpws4" Jul 13 17:05:06.254625 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.154033 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2qvc\" (UniqueName: \"kubernetes.io/projected/a7a4b481-50e3-4bb3-bcb5-6c5942814726-kube-api-access-z2qvc\") pod \"node-resolver-qn7kc\" (UID: \"a7a4b481-50e3-4bb3-bcb5-6c5942814726\") " pod="openshift-dns/node-resolver-qn7kc" Jul 13 17:05:06.254625 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.154068 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/252eeac8-ef53-439f-8d2f-8ef03a102104-multus-socket-dir-parent\") pod \"multus-cwgjn\" (UID: \"252eeac8-ef53-439f-8d2f-8ef03a102104\") " pod="openshift-multus/multus-cwgjn" Jul 13 17:05:06.254625 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.154125 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jt287\" (UniqueName: \"kubernetes.io/projected/252eeac8-ef53-439f-8d2f-8ef03a102104-kube-api-access-jt287\") pod \"multus-cwgjn\" (UID: \"252eeac8-ef53-439f-8d2f-8ef03a102104\") " pod="openshift-multus/multus-cwgjn" Jul 13 17:05:06.254625 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.154160 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6a531b70-4a58-4752-8fa8-5ba634b542ea-systemd-units\") pod \"ovnkube-node-q2pnb\" (UID: \"6a531b70-4a58-4752-8fa8-5ba634b542ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.254625 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.154198 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6a531b70-4a58-4752-8fa8-5ba634b542ea-etc-openvswitch\") pod \"ovnkube-node-q2pnb\" (UID: \"6a531b70-4a58-4752-8fa8-5ba634b542ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.254625 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.154227 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/921a5cfd-83c7-4db7-be13-53e44c5df655-cni-binary-copy\") pod \"multus-additional-cni-plugins-xpws4\" (UID: \"921a5cfd-83c7-4db7-be13-53e44c5df655\") " pod="openshift-multus/multus-additional-cni-plugins-xpws4" Jul 13 17:05:06.254625 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.154262 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/921a5cfd-83c7-4db7-be13-53e44c5df655-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xpws4\" (UID: \"921a5cfd-83c7-4db7-be13-53e44c5df655\") " pod="openshift-multus/multus-additional-cni-plugins-xpws4" Jul 13 17:05:06.254625 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.154296 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdxvt\" (UniqueName: \"kubernetes.io/projected/6c4921ee-5d95-4ec9-b2df-e42ae98f491a-kube-api-access-gdxvt\") pod \"aws-ebs-csi-driver-node-c7g2s\" (UID: \"6c4921ee-5d95-4ec9-b2df-e42ae98f491a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7g2s" Jul 13 17:05:06.254625 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.154358 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/252eeac8-ef53-439f-8d2f-8ef03a102104-os-release\") pod \"multus-cwgjn\" (UID: \"252eeac8-ef53-439f-8d2f-8ef03a102104\") " pod="openshift-multus/multus-cwgjn" Jul 13 17:05:06.254625 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.154361 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-220.ec2.internal" event={"ID":"45606745bd8d4c83782613a3e901378b","Type":"ContainerStarted","Data":"58c6a41a521cef6294ca68dacf1555ad293426f5d4640b795404fe690da28dba"} Jul 13 17:05:06.254625 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.154385 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/252eeac8-ef53-439f-8d2f-8ef03a102104-host-var-lib-cni-bin\") pod \"multus-cwgjn\" (UID: \"252eeac8-ef53-439f-8d2f-8ef03a102104\") " pod="openshift-multus/multus-cwgjn" Jul 13 17:05:06.254625 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.154436 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/252eeac8-ef53-439f-8d2f-8ef03a102104-host-var-lib-cni-bin\") pod \"multus-cwgjn\" (UID: \"252eeac8-ef53-439f-8d2f-8ef03a102104\") " pod="openshift-multus/multus-cwgjn" Jul 13 17:05:06.254625 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.154458 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b3ec2a99-eadb-4865-ae2a-dd77aab461de-konnectivity-ca\") pod \"konnectivity-agent-phcxr\" (UID: \"b3ec2a99-eadb-4865-ae2a-dd77aab461de\") " pod="kube-system/konnectivity-agent-phcxr" Jul 13 17:05:06.254625 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.154476 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/252eeac8-ef53-439f-8d2f-8ef03a102104-hostroot\") pod \"multus-cwgjn\" (UID: \"252eeac8-ef53-439f-8d2f-8ef03a102104\") " pod="openshift-multus/multus-cwgjn" Jul 13 17:05:06.254625 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.154512 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6a531b70-4a58-4752-8fa8-5ba634b542ea-systemd-units\") pod \"ovnkube-node-q2pnb\" (UID: \"6a531b70-4a58-4752-8fa8-5ba634b542ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.254625 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.154642 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6a531b70-4a58-4752-8fa8-5ba634b542ea-etc-openvswitch\") pod \"ovnkube-node-q2pnb\" (UID: \"6a531b70-4a58-4752-8fa8-5ba634b542ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.254625 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.154803 2569 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 13 17:05:06.255863 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.154831 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/252eeac8-ef53-439f-8d2f-8ef03a102104-multus-socket-dir-parent\") pod \"multus-cwgjn\" (UID: \"252eeac8-ef53-439f-8d2f-8ef03a102104\") " pod="openshift-multus/multus-cwgjn" Jul 13 17:05:06.255863 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.154916 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/252eeac8-ef53-439f-8d2f-8ef03a102104-hostroot\") pod \"multus-cwgjn\" (UID: \"252eeac8-ef53-439f-8d2f-8ef03a102104\") " pod="openshift-multus/multus-cwgjn" Jul 13 17:05:06.255863 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.154984 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/252eeac8-ef53-439f-8d2f-8ef03a102104-etc-kubernetes\") pod \"multus-cwgjn\" (UID: \"252eeac8-ef53-439f-8d2f-8ef03a102104\") " pod="openshift-multus/multus-cwgjn" Jul 13 17:05:06.255863 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.155072 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/252eeac8-ef53-439f-8d2f-8ef03a102104-os-release\") pod \"multus-cwgjn\" (UID: \"252eeac8-ef53-439f-8d2f-8ef03a102104\") " pod="openshift-multus/multus-cwgjn" Jul 13 17:05:06.255863 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.155071 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w27sn\" (UniqueName: \"kubernetes.io/projected/d401c2d6-40c6-4728-a35c-1efd3f17bb9f-kube-api-access-w27sn\") pod \"iptables-alerter-tnc2p\" (UID: \"d401c2d6-40c6-4728-a35c-1efd3f17bb9f\") " pod="openshift-network-operator/iptables-alerter-tnc2p" Jul 13 17:05:06.255863 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.155172 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/921a5cfd-83c7-4db7-be13-53e44c5df655-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xpws4\" (UID: \"921a5cfd-83c7-4db7-be13-53e44c5df655\") " pod="openshift-multus/multus-additional-cni-plugins-xpws4" Jul 13 17:05:06.255863 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.155292 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d401c2d6-40c6-4728-a35c-1efd3f17bb9f-host-slash\") pod \"iptables-alerter-tnc2p\" (UID: \"d401c2d6-40c6-4728-a35c-1efd3f17bb9f\") " pod="openshift-network-operator/iptables-alerter-tnc2p" Jul 13 17:05:06.255863 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.155379 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/252eeac8-ef53-439f-8d2f-8ef03a102104-etc-kubernetes\") pod \"multus-cwgjn\" (UID: \"252eeac8-ef53-439f-8d2f-8ef03a102104\") " pod="openshift-multus/multus-cwgjn" Jul 13 17:05:06.255863 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.155623 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/921a5cfd-83c7-4db7-be13-53e44c5df655-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xpws4\" (UID: \"921a5cfd-83c7-4db7-be13-53e44c5df655\") " pod="openshift-multus/multus-additional-cni-plugins-xpws4" Jul 13 17:05:06.255863 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.155759 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/921a5cfd-83c7-4db7-be13-53e44c5df655-cni-binary-copy\") pod \"multus-additional-cni-plugins-xpws4\" (UID: \"921a5cfd-83c7-4db7-be13-53e44c5df655\") " pod="openshift-multus/multus-additional-cni-plugins-xpws4" Jul 13 17:05:06.255863 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.155336 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6c4921ee-5d95-4ec9-b2df-e42ae98f491a-registration-dir\") pod \"aws-ebs-csi-driver-node-c7g2s\" (UID: \"6c4921ee-5d95-4ec9-b2df-e42ae98f491a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7g2s" Jul 13 17:05:06.255863 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.155847 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/252eeac8-ef53-439f-8d2f-8ef03a102104-system-cni-dir\") pod \"multus-cwgjn\" (UID: \"252eeac8-ef53-439f-8d2f-8ef03a102104\") " pod="openshift-multus/multus-cwgjn" Jul 13 17:05:06.255863 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.156216 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/252eeac8-ef53-439f-8d2f-8ef03a102104-system-cni-dir\") pod \"multus-cwgjn\" (UID: \"252eeac8-ef53-439f-8d2f-8ef03a102104\") " pod="openshift-multus/multus-cwgjn" Jul 13 17:05:06.255863 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.156233 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6a531b70-4a58-4752-8fa8-5ba634b542ea-run-systemd\") pod \"ovnkube-node-q2pnb\" (UID: \"6a531b70-4a58-4752-8fa8-5ba634b542ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.255863 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.156267 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f4054b54-56fb-46d9-8302-2618330b5e9a-etc-sysconfig\") pod \"tuned-fhjzk\" (UID: \"f4054b54-56fb-46d9-8302-2618330b5e9a\") " pod="openshift-cluster-node-tuning-operator/tuned-fhjzk" Jul 13 17:05:06.255863 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.156325 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6a531b70-4a58-4752-8fa8-5ba634b542ea-run-systemd\") pod \"ovnkube-node-q2pnb\" (UID: \"6a531b70-4a58-4752-8fa8-5ba634b542ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.255863 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.156354 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2263bf98-5f88-41d9-b755-9f874a3d5916-kubelet-config\") pod \"global-pull-secret-syncer-nfjpb\" (UID: \"2263bf98-5f88-41d9-b755-9f874a3d5916\") " pod="kube-system/global-pull-secret-syncer-nfjpb" Jul 13 17:05:06.256484 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.156378 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a7a4b481-50e3-4bb3-bcb5-6c5942814726-hosts-file\") pod \"node-resolver-qn7kc\" (UID: \"a7a4b481-50e3-4bb3-bcb5-6c5942814726\") " pod="openshift-dns/node-resolver-qn7kc" Jul 13 17:05:06.256484 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.156401 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/252eeac8-ef53-439f-8d2f-8ef03a102104-cni-binary-copy\") pod \"multus-cwgjn\" (UID: \"252eeac8-ef53-439f-8d2f-8ef03a102104\") " pod="openshift-multus/multus-cwgjn" Jul 13 17:05:06.256484 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.156414 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2263bf98-5f88-41d9-b755-9f874a3d5916-kubelet-config\") pod \"global-pull-secret-syncer-nfjpb\" (UID: \"2263bf98-5f88-41d9-b755-9f874a3d5916\") " pod="kube-system/global-pull-secret-syncer-nfjpb" Jul 13 17:05:06.256484 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.156425 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gx6wq\" (UniqueName: \"kubernetes.io/projected/7cb8e14a-393f-43f2-b75a-7f14756cfb2a-kube-api-access-gx6wq\") pod \"network-metrics-daemon-t6r98\" (UID: \"7cb8e14a-393f-43f2-b75a-7f14756cfb2a\") " pod="openshift-multus/network-metrics-daemon-t6r98" Jul 13 17:05:06.256484 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.156455 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6a531b70-4a58-4752-8fa8-5ba634b542ea-host-cni-bin\") pod \"ovnkube-node-q2pnb\" (UID: \"6a531b70-4a58-4752-8fa8-5ba634b542ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.256484 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.156477 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6a531b70-4a58-4752-8fa8-5ba634b542ea-env-overrides\") pod \"ovnkube-node-q2pnb\" (UID: \"6a531b70-4a58-4752-8fa8-5ba634b542ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.256484 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.156501 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6c4921ee-5d95-4ec9-b2df-e42ae98f491a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c7g2s\" (UID: \"6c4921ee-5d95-4ec9-b2df-e42ae98f491a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7g2s" Jul 13 17:05:06.256484 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.156525 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f4054b54-56fb-46d9-8302-2618330b5e9a-etc-sysctl-d\") pod \"tuned-fhjzk\" (UID: \"f4054b54-56fb-46d9-8302-2618330b5e9a\") " pod="openshift-cluster-node-tuning-operator/tuned-fhjzk" Jul 13 17:05:06.256484 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.156536 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6a531b70-4a58-4752-8fa8-5ba634b542ea-host-cni-bin\") pod \"ovnkube-node-q2pnb\" (UID: \"6a531b70-4a58-4752-8fa8-5ba634b542ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.256484 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.156548 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/252eeac8-ef53-439f-8d2f-8ef03a102104-host-var-lib-kubelet\") pod \"multus-cwgjn\" (UID: \"252eeac8-ef53-439f-8d2f-8ef03a102104\") " pod="openshift-multus/multus-cwgjn" Jul 13 17:05:06.256484 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.156587 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a531b70-4a58-4752-8fa8-5ba634b542ea-host-kubelet\") pod \"ovnkube-node-q2pnb\" (UID: \"6a531b70-4a58-4752-8fa8-5ba634b542ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.256484 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.156611 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6a531b70-4a58-4752-8fa8-5ba634b542ea-log-socket\") pod \"ovnkube-node-q2pnb\" (UID: \"6a531b70-4a58-4752-8fa8-5ba634b542ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.256484 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.156633 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d401c2d6-40c6-4728-a35c-1efd3f17bb9f-iptables-alerter-script\") pod \"iptables-alerter-tnc2p\" (UID: \"d401c2d6-40c6-4728-a35c-1efd3f17bb9f\") " pod="openshift-network-operator/iptables-alerter-tnc2p" Jul 13 17:05:06.256484 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.156654 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/99b9c479-5323-44a6-bbfa-2aa301efb6af-host\") pod \"node-ca-psn5f\" (UID: \"99b9c479-5323-44a6-bbfa-2aa301efb6af\") " pod="openshift-image-registry/node-ca-psn5f" Jul 13 17:05:06.256484 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.156676 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/921a5cfd-83c7-4db7-be13-53e44c5df655-cnibin\") pod \"multus-additional-cni-plugins-xpws4\" (UID: \"921a5cfd-83c7-4db7-be13-53e44c5df655\") " pod="openshift-multus/multus-additional-cni-plugins-xpws4" Jul 13 17:05:06.256484 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.156698 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-869n2\" (UniqueName: \"kubernetes.io/projected/921a5cfd-83c7-4db7-be13-53e44c5df655-kube-api-access-869n2\") pod \"multus-additional-cni-plugins-xpws4\" (UID: \"921a5cfd-83c7-4db7-be13-53e44c5df655\") " pod="openshift-multus/multus-additional-cni-plugins-xpws4" Jul 13 17:05:06.256484 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.156719 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f4054b54-56fb-46d9-8302-2618330b5e9a-sys\") pod \"tuned-fhjzk\" (UID: \"f4054b54-56fb-46d9-8302-2618330b5e9a\") " pod="openshift-cluster-node-tuning-operator/tuned-fhjzk" Jul 13 17:05:06.257461 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.156742 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2263bf98-5f88-41d9-b755-9f874a3d5916-dbus\") pod \"global-pull-secret-syncer-nfjpb\" (UID: \"2263bf98-5f88-41d9-b755-9f874a3d5916\") " pod="kube-system/global-pull-secret-syncer-nfjpb" Jul 13 17:05:06.257461 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.156765 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a531b70-4a58-4752-8fa8-5ba634b542ea-host-run-ovn-kubernetes\") pod \"ovnkube-node-q2pnb\" (UID: \"6a531b70-4a58-4752-8fa8-5ba634b542ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.257461 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.156800 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6a531b70-4a58-4752-8fa8-5ba634b542ea-ovnkube-script-lib\") pod \"ovnkube-node-q2pnb\" (UID: \"6a531b70-4a58-4752-8fa8-5ba634b542ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.257461 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.156820 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f4054b54-56fb-46d9-8302-2618330b5e9a-etc-tuned\") pod \"tuned-fhjzk\" (UID: \"f4054b54-56fb-46d9-8302-2618330b5e9a\") " pod="openshift-cluster-node-tuning-operator/tuned-fhjzk" Jul 13 17:05:06.257461 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.156843 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/252eeac8-ef53-439f-8d2f-8ef03a102104-multus-conf-dir\") pod \"multus-cwgjn\" (UID: \"252eeac8-ef53-439f-8d2f-8ef03a102104\") " pod="openshift-multus/multus-cwgjn" Jul 13 17:05:06.257461 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.156875 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/252eeac8-ef53-439f-8d2f-8ef03a102104-multus-daemon-config\") pod \"multus-cwgjn\" (UID: \"252eeac8-ef53-439f-8d2f-8ef03a102104\") " pod="openshift-multus/multus-cwgjn" Jul 13 17:05:06.257461 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.156900 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a531b70-4a58-4752-8fa8-5ba634b542ea-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-q2pnb\" (UID: \"6a531b70-4a58-4752-8fa8-5ba634b542ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.257461 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.156917 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-220.ec2.internal" event={"ID":"d00899e62aada42f2f0bdc458167583d","Type":"ContainerStarted","Data":"ae76710ebbfbe227d2d77a7f16f87c70c644ebf1c2f7ee8621e021d30a432607"} Jul 13 17:05:06.257461 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.156923 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6c4921ee-5d95-4ec9-b2df-e42ae98f491a-socket-dir\") pod \"aws-ebs-csi-driver-node-c7g2s\" (UID: \"6c4921ee-5d95-4ec9-b2df-e42ae98f491a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7g2s" Jul 13 17:05:06.257461 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.156951 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/99b9c479-5323-44a6-bbfa-2aa301efb6af-host\") pod \"node-ca-psn5f\" (UID: \"99b9c479-5323-44a6-bbfa-2aa301efb6af\") " pod="openshift-image-registry/node-ca-psn5f" Jul 13 17:05:06.257461 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.156959 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6c4921ee-5d95-4ec9-b2df-e42ae98f491a-sys-fs\") pod \"aws-ebs-csi-driver-node-c7g2s\" (UID: \"6c4921ee-5d95-4ec9-b2df-e42ae98f491a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7g2s" Jul 13 17:05:06.257461 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.157000 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/921a5cfd-83c7-4db7-be13-53e44c5df655-cnibin\") pod \"multus-additional-cni-plugins-xpws4\" (UID: \"921a5cfd-83c7-4db7-be13-53e44c5df655\") " pod="openshift-multus/multus-additional-cni-plugins-xpws4" Jul 13 17:05:06.257461 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.156499 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a7a4b481-50e3-4bb3-bcb5-6c5942814726-hosts-file\") pod \"node-resolver-qn7kc\" (UID: \"a7a4b481-50e3-4bb3-bcb5-6c5942814726\") " pod="openshift-dns/node-resolver-qn7kc" Jul 13 17:05:06.257461 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.157054 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/252eeac8-ef53-439f-8d2f-8ef03a102104-cni-binary-copy\") pod \"multus-cwgjn\" (UID: \"252eeac8-ef53-439f-8d2f-8ef03a102104\") " pod="openshift-multus/multus-cwgjn" Jul 13 17:05:06.257461 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.157063 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a531b70-4a58-4752-8fa8-5ba634b542ea-host-kubelet\") pod \"ovnkube-node-q2pnb\" (UID: \"6a531b70-4a58-4752-8fa8-5ba634b542ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.257461 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.157074 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/252eeac8-ef53-439f-8d2f-8ef03a102104-host-var-lib-kubelet\") pod \"multus-cwgjn\" (UID: \"252eeac8-ef53-439f-8d2f-8ef03a102104\") " pod="openshift-multus/multus-cwgjn" Jul 13 17:05:06.257461 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.157107 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a531b70-4a58-4752-8fa8-5ba634b542ea-host-run-ovn-kubernetes\") pod \"ovnkube-node-q2pnb\" (UID: \"6a531b70-4a58-4752-8fa8-5ba634b542ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.258214 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.157119 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2263bf98-5f88-41d9-b755-9f874a3d5916-dbus\") pod \"global-pull-secret-syncer-nfjpb\" (UID: \"2263bf98-5f88-41d9-b755-9f874a3d5916\") " pod="kube-system/global-pull-secret-syncer-nfjpb" Jul 13 17:05:06.258214 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.157151 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/252eeac8-ef53-439f-8d2f-8ef03a102104-multus-conf-dir\") pod \"multus-cwgjn\" (UID: \"252eeac8-ef53-439f-8d2f-8ef03a102104\") " pod="openshift-multus/multus-cwgjn" Jul 13 17:05:06.258214 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.157182 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6a531b70-4a58-4752-8fa8-5ba634b542ea-log-socket\") pod \"ovnkube-node-q2pnb\" (UID: \"6a531b70-4a58-4752-8fa8-5ba634b542ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.258214 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.157383 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6a531b70-4a58-4752-8fa8-5ba634b542ea-env-overrides\") pod \"ovnkube-node-q2pnb\" (UID: \"6a531b70-4a58-4752-8fa8-5ba634b542ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.258214 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.157431 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f4054b54-56fb-46d9-8302-2618330b5e9a-etc-sysctl-conf\") pod \"tuned-fhjzk\" (UID: \"f4054b54-56fb-46d9-8302-2618330b5e9a\") " pod="openshift-cluster-node-tuning-operator/tuned-fhjzk" Jul 13 17:05:06.258214 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.157462 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/252eeac8-ef53-439f-8d2f-8ef03a102104-host-var-lib-cni-multus\") pod \"multus-cwgjn\" (UID: \"252eeac8-ef53-439f-8d2f-8ef03a102104\") " pod="openshift-multus/multus-cwgjn" Jul 13 17:05:06.258214 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.157487 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7a4b481-50e3-4bb3-bcb5-6c5942814726-tmp-dir\") pod \"node-resolver-qn7kc\" (UID: \"a7a4b481-50e3-4bb3-bcb5-6c5942814726\") " pod="openshift-dns/node-resolver-qn7kc" Jul 13 17:05:06.258214 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.157512 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6a531b70-4a58-4752-8fa8-5ba634b542ea-host-cni-netd\") pod \"ovnkube-node-q2pnb\" (UID: \"6a531b70-4a58-4752-8fa8-5ba634b542ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.258214 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.157529 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/252eeac8-ef53-439f-8d2f-8ef03a102104-multus-daemon-config\") pod \"multus-cwgjn\" (UID: \"252eeac8-ef53-439f-8d2f-8ef03a102104\") " pod="openshift-multus/multus-cwgjn" Jul 13 17:05:06.258214 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.157556 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6c4921ee-5d95-4ec9-b2df-e42ae98f491a-device-dir\") pod \"aws-ebs-csi-driver-node-c7g2s\" (UID: \"6c4921ee-5d95-4ec9-b2df-e42ae98f491a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7g2s" Jul 13 17:05:06.258214 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.157598 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6a531b70-4a58-4752-8fa8-5ba634b542ea-host-cni-netd\") pod \"ovnkube-node-q2pnb\" (UID: \"6a531b70-4a58-4752-8fa8-5ba634b542ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.258214 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.157600 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f4054b54-56fb-46d9-8302-2618330b5e9a-etc-modprobe-d\") pod \"tuned-fhjzk\" (UID: \"f4054b54-56fb-46d9-8302-2618330b5e9a\") " pod="openshift-cluster-node-tuning-operator/tuned-fhjzk" Jul 13 17:05:06.258214 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.157634 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/252eeac8-ef53-439f-8d2f-8ef03a102104-host-var-lib-cni-multus\") pod \"multus-cwgjn\" (UID: \"252eeac8-ef53-439f-8d2f-8ef03a102104\") " pod="openshift-multus/multus-cwgjn" Jul 13 17:05:06.258214 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.157637 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f4054b54-56fb-46d9-8302-2618330b5e9a-run\") pod \"tuned-fhjzk\" (UID: \"f4054b54-56fb-46d9-8302-2618330b5e9a\") " pod="openshift-cluster-node-tuning-operator/tuned-fhjzk" Jul 13 17:05:06.258214 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.157671 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/252eeac8-ef53-439f-8d2f-8ef03a102104-cnibin\") pod \"multus-cwgjn\" (UID: \"252eeac8-ef53-439f-8d2f-8ef03a102104\") " pod="openshift-multus/multus-cwgjn" Jul 13 17:05:06.258214 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.157689 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d401c2d6-40c6-4728-a35c-1efd3f17bb9f-iptables-alerter-script\") pod \"iptables-alerter-tnc2p\" (UID: \"d401c2d6-40c6-4728-a35c-1efd3f17bb9f\") " pod="openshift-network-operator/iptables-alerter-tnc2p" Jul 13 17:05:06.258214 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.157730 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/252eeac8-ef53-439f-8d2f-8ef03a102104-host-run-k8s-cni-cncf-io\") pod \"multus-cwgjn\" (UID: \"252eeac8-ef53-439f-8d2f-8ef03a102104\") " pod="openshift-multus/multus-cwgjn" Jul 13 17:05:06.258214 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.157734 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6a531b70-4a58-4752-8fa8-5ba634b542ea-ovnkube-script-lib\") pod \"ovnkube-node-q2pnb\" (UID: \"6a531b70-4a58-4752-8fa8-5ba634b542ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.259033 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.157748 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a531b70-4a58-4752-8fa8-5ba634b542ea-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-q2pnb\" (UID: \"6a531b70-4a58-4752-8fa8-5ba634b542ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.259033 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.157757 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7cb8e14a-393f-43f2-b75a-7f14756cfb2a-metrics-certs\") pod \"network-metrics-daemon-t6r98\" (UID: \"7cb8e14a-393f-43f2-b75a-7f14756cfb2a\") " pod="openshift-multus/network-metrics-daemon-t6r98" Jul 13 17:05:06.259033 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.157796 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6a531b70-4a58-4752-8fa8-5ba634b542ea-host-run-netns\") pod \"ovnkube-node-q2pnb\" (UID: \"6a531b70-4a58-4752-8fa8-5ba634b542ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.259033 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.157805 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/252eeac8-ef53-439f-8d2f-8ef03a102104-cnibin\") pod \"multus-cwgjn\" (UID: \"252eeac8-ef53-439f-8d2f-8ef03a102104\") " pod="openshift-multus/multus-cwgjn" Jul 13 17:05:06.259033 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:06.157870 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 17:05:06.259033 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.157893 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/252eeac8-ef53-439f-8d2f-8ef03a102104-host-run-k8s-cni-cncf-io\") pod \"multus-cwgjn\" (UID: \"252eeac8-ef53-439f-8d2f-8ef03a102104\") " pod="openshift-multus/multus-cwgjn" Jul 13 17:05:06.259033 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.157895 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6c4921ee-5d95-4ec9-b2df-e42ae98f491a-etc-selinux\") pod \"aws-ebs-csi-driver-node-c7g2s\" (UID: \"6c4921ee-5d95-4ec9-b2df-e42ae98f491a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7g2s" Jul 13 17:05:06.259033 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:06.157915 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cb8e14a-393f-43f2-b75a-7f14756cfb2a-metrics-certs podName:7cb8e14a-393f-43f2-b75a-7f14756cfb2a nodeName:}" failed. No retries permitted until 2026-07-13 17:05:06.65790001 +0000 UTC m=+3.055558369 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7cb8e14a-393f-43f2-b75a-7f14756cfb2a-metrics-certs") pod "network-metrics-daemon-t6r98" (UID: "7cb8e14a-393f-43f2-b75a-7f14756cfb2a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 17:05:06.259033 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.157927 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6a531b70-4a58-4752-8fa8-5ba634b542ea-host-run-netns\") pod \"ovnkube-node-q2pnb\" (UID: \"6a531b70-4a58-4752-8fa8-5ba634b542ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.259033 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.157958 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f4054b54-56fb-46d9-8302-2618330b5e9a-etc-kubernetes\") pod \"tuned-fhjzk\" (UID: \"f4054b54-56fb-46d9-8302-2618330b5e9a\") " pod="openshift-cluster-node-tuning-operator/tuned-fhjzk" Jul 13 17:05:06.259033 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.157995 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6a531b70-4a58-4752-8fa8-5ba634b542ea-run-openvswitch\") pod \"ovnkube-node-q2pnb\" (UID: \"6a531b70-4a58-4752-8fa8-5ba634b542ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.259033 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.158023 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6a531b70-4a58-4752-8fa8-5ba634b542ea-node-log\") pod \"ovnkube-node-q2pnb\" (UID: \"6a531b70-4a58-4752-8fa8-5ba634b542ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.259033 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.158057 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6a531b70-4a58-4752-8fa8-5ba634b542ea-ovn-node-metrics-cert\") pod \"ovnkube-node-q2pnb\" (UID: \"6a531b70-4a58-4752-8fa8-5ba634b542ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.259033 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.158115 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6a531b70-4a58-4752-8fa8-5ba634b542ea-node-log\") pod \"ovnkube-node-q2pnb\" (UID: \"6a531b70-4a58-4752-8fa8-5ba634b542ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.259033 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.158144 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6a531b70-4a58-4752-8fa8-5ba634b542ea-run-openvswitch\") pod \"ovnkube-node-q2pnb\" (UID: \"6a531b70-4a58-4752-8fa8-5ba634b542ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.259033 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.158155 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/99b9c479-5323-44a6-bbfa-2aa301efb6af-serviceca\") pod \"node-ca-psn5f\" (UID: \"99b9c479-5323-44a6-bbfa-2aa301efb6af\") " pod="openshift-image-registry/node-ca-psn5f" Jul 13 17:05:06.259033 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.158184 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7a4b481-50e3-4bb3-bcb5-6c5942814726-tmp-dir\") pod \"node-resolver-qn7kc\" (UID: \"a7a4b481-50e3-4bb3-bcb5-6c5942814726\") " pod="openshift-dns/node-resolver-qn7kc" Jul 13 17:05:06.259860 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.158191 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4w5m\" (UniqueName: \"kubernetes.io/projected/99b9c479-5323-44a6-bbfa-2aa301efb6af-kube-api-access-n4w5m\") pod \"node-ca-psn5f\" (UID: \"99b9c479-5323-44a6-bbfa-2aa301efb6af\") " pod="openshift-image-registry/node-ca-psn5f" Jul 13 17:05:06.259860 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.158291 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/921a5cfd-83c7-4db7-be13-53e44c5df655-system-cni-dir\") pod \"multus-additional-cni-plugins-xpws4\" (UID: \"921a5cfd-83c7-4db7-be13-53e44c5df655\") " pod="openshift-multus/multus-additional-cni-plugins-xpws4" Jul 13 17:05:06.259860 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.158401 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/921a5cfd-83c7-4db7-be13-53e44c5df655-os-release\") pod \"multus-additional-cni-plugins-xpws4\" (UID: \"921a5cfd-83c7-4db7-be13-53e44c5df655\") " pod="openshift-multus/multus-additional-cni-plugins-xpws4" Jul 13 17:05:06.259860 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.158431 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f4054b54-56fb-46d9-8302-2618330b5e9a-lib-modules\") pod \"tuned-fhjzk\" (UID: \"f4054b54-56fb-46d9-8302-2618330b5e9a\") " pod="openshift-cluster-node-tuning-operator/tuned-fhjzk" Jul 13 17:05:06.259860 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.158476 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6a531b70-4a58-4752-8fa8-5ba634b542ea-host-slash\") pod \"ovnkube-node-q2pnb\" (UID: \"6a531b70-4a58-4752-8fa8-5ba634b542ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.259860 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.158501 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/921a5cfd-83c7-4db7-be13-53e44c5df655-os-release\") pod \"multus-additional-cni-plugins-xpws4\" (UID: \"921a5cfd-83c7-4db7-be13-53e44c5df655\") " pod="openshift-multus/multus-additional-cni-plugins-xpws4" Jul 13 17:05:06.259860 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.158503 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/921a5cfd-83c7-4db7-be13-53e44c5df655-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xpws4\" (UID: \"921a5cfd-83c7-4db7-be13-53e44c5df655\") " pod="openshift-multus/multus-additional-cni-plugins-xpws4" Jul 13 17:05:06.259860 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.158543 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f4054b54-56fb-46d9-8302-2618330b5e9a-host\") pod \"tuned-fhjzk\" (UID: \"f4054b54-56fb-46d9-8302-2618330b5e9a\") " pod="openshift-cluster-node-tuning-operator/tuned-fhjzk" Jul 13 17:05:06.259860 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.158549 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6a531b70-4a58-4752-8fa8-5ba634b542ea-host-slash\") pod \"ovnkube-node-q2pnb\" (UID: \"6a531b70-4a58-4752-8fa8-5ba634b542ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.259860 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.158581 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/99b9c479-5323-44a6-bbfa-2aa301efb6af-serviceca\") pod \"node-ca-psn5f\" (UID: \"99b9c479-5323-44a6-bbfa-2aa301efb6af\") " pod="openshift-image-registry/node-ca-psn5f" Jul 13 17:05:06.259860 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.158596 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/921a5cfd-83c7-4db7-be13-53e44c5df655-system-cni-dir\") pod \"multus-additional-cni-plugins-xpws4\" (UID: \"921a5cfd-83c7-4db7-be13-53e44c5df655\") " pod="openshift-multus/multus-additional-cni-plugins-xpws4" Jul 13 17:05:06.259860 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.158644 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/252eeac8-ef53-439f-8d2f-8ef03a102104-host-run-multus-certs\") pod \"multus-cwgjn\" (UID: \"252eeac8-ef53-439f-8d2f-8ef03a102104\") " pod="openshift-multus/multus-cwgjn" Jul 13 17:05:06.259860 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.158679 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/252eeac8-ef53-439f-8d2f-8ef03a102104-host-run-multus-certs\") pod \"multus-cwgjn\" (UID: \"252eeac8-ef53-439f-8d2f-8ef03a102104\") " pod="openshift-multus/multus-cwgjn" Jul 13 17:05:06.259860 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.158699 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6a531b70-4a58-4752-8fa8-5ba634b542ea-var-lib-openvswitch\") pod \"ovnkube-node-q2pnb\" (UID: \"6a531b70-4a58-4752-8fa8-5ba634b542ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.259860 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.158730 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6a531b70-4a58-4752-8fa8-5ba634b542ea-ovnkube-config\") pod \"ovnkube-node-q2pnb\" (UID: \"6a531b70-4a58-4752-8fa8-5ba634b542ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.259860 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.158761 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6a531b70-4a58-4752-8fa8-5ba634b542ea-var-lib-openvswitch\") pod \"ovnkube-node-q2pnb\" (UID: \"6a531b70-4a58-4752-8fa8-5ba634b542ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.259860 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.158787 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75wpl\" (UniqueName: \"kubernetes.io/projected/6a531b70-4a58-4752-8fa8-5ba634b542ea-kube-api-access-75wpl\") pod \"ovnkube-node-q2pnb\" (UID: \"6a531b70-4a58-4752-8fa8-5ba634b542ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.260797 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.159117 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b3ec2a99-eadb-4865-ae2a-dd77aab461de-agent-certs\") pod \"konnectivity-agent-phcxr\" (UID: \"b3ec2a99-eadb-4865-ae2a-dd77aab461de\") " pod="kube-system/konnectivity-agent-phcxr" Jul 13 17:05:06.260797 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.159195 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6a531b70-4a58-4752-8fa8-5ba634b542ea-ovnkube-config\") pod \"ovnkube-node-q2pnb\" (UID: \"6a531b70-4a58-4752-8fa8-5ba634b542ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.260797 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.159540 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/921a5cfd-83c7-4db7-be13-53e44c5df655-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xpws4\" (UID: \"921a5cfd-83c7-4db7-be13-53e44c5df655\") " pod="openshift-multus/multus-additional-cni-plugins-xpws4" Jul 13 17:05:06.260797 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.160360 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6a531b70-4a58-4752-8fa8-5ba634b542ea-ovn-node-metrics-cert\") pod \"ovnkube-node-q2pnb\" (UID: \"6a531b70-4a58-4752-8fa8-5ba634b542ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.260797 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:06.163706 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 17:05:06.260797 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:06.163743 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 17:05:06.260797 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:06.163755 2569 projected.go:196] Error preparing data for projected volume kube-api-access-z6hpn for pod openshift-network-diagnostics/network-check-target-2b6wx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 17:05:06.260797 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:06.163808 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5d1dd68a-166a-4be1-812e-3e15e056c3f2-kube-api-access-z6hpn podName:5d1dd68a-166a-4be1-812e-3e15e056c3f2 nodeName:}" failed. No retries permitted until 2026-07-13 17:05:06.663794937 +0000 UTC m=+3.061453294 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-z6hpn" (UniqueName: "kubernetes.io/projected/5d1dd68a-166a-4be1-812e-3e15e056c3f2-kube-api-access-z6hpn") pod "network-check-target-2b6wx" (UID: "5d1dd68a-166a-4be1-812e-3e15e056c3f2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 17:05:06.260797 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.166335 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w27sn\" (UniqueName: \"kubernetes.io/projected/d401c2d6-40c6-4728-a35c-1efd3f17bb9f-kube-api-access-w27sn\") pod \"iptables-alerter-tnc2p\" (UID: \"d401c2d6-40c6-4728-a35c-1efd3f17bb9f\") " pod="openshift-network-operator/iptables-alerter-tnc2p" Jul 13 17:05:06.260797 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.167906 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jt287\" (UniqueName: \"kubernetes.io/projected/252eeac8-ef53-439f-8d2f-8ef03a102104-kube-api-access-jt287\") pod \"multus-cwgjn\" (UID: \"252eeac8-ef53-439f-8d2f-8ef03a102104\") " pod="openshift-multus/multus-cwgjn" Jul 13 17:05:06.260797 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.168532 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2qvc\" (UniqueName: \"kubernetes.io/projected/a7a4b481-50e3-4bb3-bcb5-6c5942814726-kube-api-access-z2qvc\") pod \"node-resolver-qn7kc\" (UID: \"a7a4b481-50e3-4bb3-bcb5-6c5942814726\") " pod="openshift-dns/node-resolver-qn7kc" Jul 13 17:05:06.260797 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.168931 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4w5m\" (UniqueName: \"kubernetes.io/projected/99b9c479-5323-44a6-bbfa-2aa301efb6af-kube-api-access-n4w5m\") pod \"node-ca-psn5f\" (UID: \"99b9c479-5323-44a6-bbfa-2aa301efb6af\") " pod="openshift-image-registry/node-ca-psn5f" Jul 13 17:05:06.260797 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.169699 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75wpl\" (UniqueName: \"kubernetes.io/projected/6a531b70-4a58-4752-8fa8-5ba634b542ea-kube-api-access-75wpl\") pod \"ovnkube-node-q2pnb\" (UID: \"6a531b70-4a58-4752-8fa8-5ba634b542ea\") " pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.260797 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.170036 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-869n2\" (UniqueName: \"kubernetes.io/projected/921a5cfd-83c7-4db7-be13-53e44c5df655-kube-api-access-869n2\") pod \"multus-additional-cni-plugins-xpws4\" (UID: \"921a5cfd-83c7-4db7-be13-53e44c5df655\") " pod="openshift-multus/multus-additional-cni-plugins-xpws4" Jul 13 17:05:06.260797 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.170483 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gx6wq\" (UniqueName: \"kubernetes.io/projected/7cb8e14a-393f-43f2-b75a-7f14756cfb2a-kube-api-access-gx6wq\") pod \"network-metrics-daemon-t6r98\" (UID: \"7cb8e14a-393f-43f2-b75a-7f14756cfb2a\") " pod="openshift-multus/network-metrics-daemon-t6r98" Jul 13 17:05:06.260797 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.259406 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdxvt\" (UniqueName: \"kubernetes.io/projected/6c4921ee-5d95-4ec9-b2df-e42ae98f491a-kube-api-access-gdxvt\") pod \"aws-ebs-csi-driver-node-c7g2s\" (UID: \"6c4921ee-5d95-4ec9-b2df-e42ae98f491a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7g2s" Jul 13 17:05:06.260797 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.259448 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6c4921ee-5d95-4ec9-b2df-e42ae98f491a-registration-dir\") pod \"aws-ebs-csi-driver-node-c7g2s\" (UID: \"6c4921ee-5d95-4ec9-b2df-e42ae98f491a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7g2s" Jul 13 17:05:06.261476 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.259475 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f4054b54-56fb-46d9-8302-2618330b5e9a-etc-sysconfig\") pod \"tuned-fhjzk\" (UID: \"f4054b54-56fb-46d9-8302-2618330b5e9a\") " pod="openshift-cluster-node-tuning-operator/tuned-fhjzk" Jul 13 17:05:06.261476 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.259508 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6c4921ee-5d95-4ec9-b2df-e42ae98f491a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c7g2s\" (UID: \"6c4921ee-5d95-4ec9-b2df-e42ae98f491a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7g2s" Jul 13 17:05:06.261476 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.259533 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f4054b54-56fb-46d9-8302-2618330b5e9a-etc-sysctl-d\") pod \"tuned-fhjzk\" (UID: \"f4054b54-56fb-46d9-8302-2618330b5e9a\") " pod="openshift-cluster-node-tuning-operator/tuned-fhjzk" Jul 13 17:05:06.261476 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.259558 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6c4921ee-5d95-4ec9-b2df-e42ae98f491a-registration-dir\") pod \"aws-ebs-csi-driver-node-c7g2s\" (UID: \"6c4921ee-5d95-4ec9-b2df-e42ae98f491a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7g2s" Jul 13 17:05:06.261476 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.259576 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f4054b54-56fb-46d9-8302-2618330b5e9a-sys\") pod \"tuned-fhjzk\" (UID: \"f4054b54-56fb-46d9-8302-2618330b5e9a\") " pod="openshift-cluster-node-tuning-operator/tuned-fhjzk" Jul 13 17:05:06.261476 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.259623 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f4054b54-56fb-46d9-8302-2618330b5e9a-etc-sysconfig\") pod \"tuned-fhjzk\" (UID: \"f4054b54-56fb-46d9-8302-2618330b5e9a\") " pod="openshift-cluster-node-tuning-operator/tuned-fhjzk" Jul 13 17:05:06.261476 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.259630 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f4054b54-56fb-46d9-8302-2618330b5e9a-sys\") pod \"tuned-fhjzk\" (UID: \"f4054b54-56fb-46d9-8302-2618330b5e9a\") " pod="openshift-cluster-node-tuning-operator/tuned-fhjzk" Jul 13 17:05:06.261476 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.259641 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f4054b54-56fb-46d9-8302-2618330b5e9a-etc-tuned\") pod \"tuned-fhjzk\" (UID: \"f4054b54-56fb-46d9-8302-2618330b5e9a\") " pod="openshift-cluster-node-tuning-operator/tuned-fhjzk" Jul 13 17:05:06.261476 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.259676 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6c4921ee-5d95-4ec9-b2df-e42ae98f491a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c7g2s\" (UID: \"6c4921ee-5d95-4ec9-b2df-e42ae98f491a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7g2s" Jul 13 17:05:06.261476 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.259684 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6c4921ee-5d95-4ec9-b2df-e42ae98f491a-socket-dir\") pod \"aws-ebs-csi-driver-node-c7g2s\" (UID: \"6c4921ee-5d95-4ec9-b2df-e42ae98f491a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7g2s" Jul 13 17:05:06.261476 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.259709 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6c4921ee-5d95-4ec9-b2df-e42ae98f491a-sys-fs\") pod \"aws-ebs-csi-driver-node-c7g2s\" (UID: \"6c4921ee-5d95-4ec9-b2df-e42ae98f491a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7g2s" Jul 13 17:05:06.261476 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.259733 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f4054b54-56fb-46d9-8302-2618330b5e9a-etc-sysctl-conf\") pod \"tuned-fhjzk\" (UID: \"f4054b54-56fb-46d9-8302-2618330b5e9a\") " pod="openshift-cluster-node-tuning-operator/tuned-fhjzk" Jul 13 17:05:06.261476 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.259743 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f4054b54-56fb-46d9-8302-2618330b5e9a-etc-sysctl-d\") pod \"tuned-fhjzk\" (UID: \"f4054b54-56fb-46d9-8302-2618330b5e9a\") " pod="openshift-cluster-node-tuning-operator/tuned-fhjzk" Jul 13 17:05:06.261476 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.259760 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6c4921ee-5d95-4ec9-b2df-e42ae98f491a-device-dir\") pod \"aws-ebs-csi-driver-node-c7g2s\" (UID: \"6c4921ee-5d95-4ec9-b2df-e42ae98f491a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7g2s" Jul 13 17:05:06.261476 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.259816 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6c4921ee-5d95-4ec9-b2df-e42ae98f491a-device-dir\") pod \"aws-ebs-csi-driver-node-c7g2s\" (UID: \"6c4921ee-5d95-4ec9-b2df-e42ae98f491a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7g2s" Jul 13 17:05:06.261476 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.259809 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6c4921ee-5d95-4ec9-b2df-e42ae98f491a-sys-fs\") pod \"aws-ebs-csi-driver-node-c7g2s\" (UID: \"6c4921ee-5d95-4ec9-b2df-e42ae98f491a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7g2s" Jul 13 17:05:06.261476 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.259846 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f4054b54-56fb-46d9-8302-2618330b5e9a-etc-modprobe-d\") pod \"tuned-fhjzk\" (UID: \"f4054b54-56fb-46d9-8302-2618330b5e9a\") " pod="openshift-cluster-node-tuning-operator/tuned-fhjzk" Jul 13 17:05:06.262273 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.259877 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f4054b54-56fb-46d9-8302-2618330b5e9a-etc-sysctl-conf\") pod \"tuned-fhjzk\" (UID: \"f4054b54-56fb-46d9-8302-2618330b5e9a\") " pod="openshift-cluster-node-tuning-operator/tuned-fhjzk" Jul 13 17:05:06.262273 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.259873 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f4054b54-56fb-46d9-8302-2618330b5e9a-run\") pod \"tuned-fhjzk\" (UID: \"f4054b54-56fb-46d9-8302-2618330b5e9a\") " pod="openshift-cluster-node-tuning-operator/tuned-fhjzk" Jul 13 17:05:06.262273 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.259829 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6c4921ee-5d95-4ec9-b2df-e42ae98f491a-socket-dir\") pod \"aws-ebs-csi-driver-node-c7g2s\" (UID: \"6c4921ee-5d95-4ec9-b2df-e42ae98f491a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7g2s" Jul 13 17:05:06.262273 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.259931 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f4054b54-56fb-46d9-8302-2618330b5e9a-run\") pod \"tuned-fhjzk\" (UID: \"f4054b54-56fb-46d9-8302-2618330b5e9a\") " pod="openshift-cluster-node-tuning-operator/tuned-fhjzk" Jul 13 17:05:06.262273 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.259937 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f4054b54-56fb-46d9-8302-2618330b5e9a-etc-modprobe-d\") pod \"tuned-fhjzk\" (UID: \"f4054b54-56fb-46d9-8302-2618330b5e9a\") " pod="openshift-cluster-node-tuning-operator/tuned-fhjzk" Jul 13 17:05:06.262273 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.259962 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6c4921ee-5d95-4ec9-b2df-e42ae98f491a-etc-selinux\") pod \"aws-ebs-csi-driver-node-c7g2s\" (UID: \"6c4921ee-5d95-4ec9-b2df-e42ae98f491a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7g2s" Jul 13 17:05:06.262273 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.259990 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f4054b54-56fb-46d9-8302-2618330b5e9a-etc-kubernetes\") pod \"tuned-fhjzk\" (UID: \"f4054b54-56fb-46d9-8302-2618330b5e9a\") " pod="openshift-cluster-node-tuning-operator/tuned-fhjzk" Jul 13 17:05:06.262273 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.260022 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f4054b54-56fb-46d9-8302-2618330b5e9a-lib-modules\") pod \"tuned-fhjzk\" (UID: \"f4054b54-56fb-46d9-8302-2618330b5e9a\") " pod="openshift-cluster-node-tuning-operator/tuned-fhjzk" Jul 13 17:05:06.262273 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.260069 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f4054b54-56fb-46d9-8302-2618330b5e9a-host\") pod \"tuned-fhjzk\" (UID: \"f4054b54-56fb-46d9-8302-2618330b5e9a\") " pod="openshift-cluster-node-tuning-operator/tuned-fhjzk" Jul 13 17:05:06.262273 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.260071 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f4054b54-56fb-46d9-8302-2618330b5e9a-etc-kubernetes\") pod \"tuned-fhjzk\" (UID: \"f4054b54-56fb-46d9-8302-2618330b5e9a\") " pod="openshift-cluster-node-tuning-operator/tuned-fhjzk" Jul 13 17:05:06.262273 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.260071 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6c4921ee-5d95-4ec9-b2df-e42ae98f491a-etc-selinux\") pod \"aws-ebs-csi-driver-node-c7g2s\" (UID: \"6c4921ee-5d95-4ec9-b2df-e42ae98f491a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7g2s" Jul 13 17:05:06.262273 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.260097 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f4054b54-56fb-46d9-8302-2618330b5e9a-var-lib-kubelet\") pod \"tuned-fhjzk\" (UID: \"f4054b54-56fb-46d9-8302-2618330b5e9a\") " pod="openshift-cluster-node-tuning-operator/tuned-fhjzk" Jul 13 17:05:06.262273 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.260124 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f4054b54-56fb-46d9-8302-2618330b5e9a-tmp\") pod \"tuned-fhjzk\" (UID: \"f4054b54-56fb-46d9-8302-2618330b5e9a\") " pod="openshift-cluster-node-tuning-operator/tuned-fhjzk" Jul 13 17:05:06.262273 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.260135 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f4054b54-56fb-46d9-8302-2618330b5e9a-host\") pod \"tuned-fhjzk\" (UID: \"f4054b54-56fb-46d9-8302-2618330b5e9a\") " pod="openshift-cluster-node-tuning-operator/tuned-fhjzk" Jul 13 17:05:06.262273 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.260164 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f4054b54-56fb-46d9-8302-2618330b5e9a-etc-systemd\") pod \"tuned-fhjzk\" (UID: \"f4054b54-56fb-46d9-8302-2618330b5e9a\") " pod="openshift-cluster-node-tuning-operator/tuned-fhjzk" Jul 13 17:05:06.262273 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.260168 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f4054b54-56fb-46d9-8302-2618330b5e9a-lib-modules\") pod \"tuned-fhjzk\" (UID: \"f4054b54-56fb-46d9-8302-2618330b5e9a\") " pod="openshift-cluster-node-tuning-operator/tuned-fhjzk" Jul 13 17:05:06.262273 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.260167 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f4054b54-56fb-46d9-8302-2618330b5e9a-var-lib-kubelet\") pod \"tuned-fhjzk\" (UID: \"f4054b54-56fb-46d9-8302-2618330b5e9a\") " pod="openshift-cluster-node-tuning-operator/tuned-fhjzk" Jul 13 17:05:06.262273 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.260253 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f4054b54-56fb-46d9-8302-2618330b5e9a-etc-systemd\") pod \"tuned-fhjzk\" (UID: \"f4054b54-56fb-46d9-8302-2618330b5e9a\") " pod="openshift-cluster-node-tuning-operator/tuned-fhjzk" Jul 13 17:05:06.263088 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.260299 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxp6n\" (UniqueName: \"kubernetes.io/projected/f4054b54-56fb-46d9-8302-2618330b5e9a-kube-api-access-bxp6n\") pod \"tuned-fhjzk\" (UID: \"f4054b54-56fb-46d9-8302-2618330b5e9a\") " pod="openshift-cluster-node-tuning-operator/tuned-fhjzk" Jul 13 17:05:06.263088 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.262380 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f4054b54-56fb-46d9-8302-2618330b5e9a-etc-tuned\") pod \"tuned-fhjzk\" (UID: \"f4054b54-56fb-46d9-8302-2618330b5e9a\") " pod="openshift-cluster-node-tuning-operator/tuned-fhjzk" Jul 13 17:05:06.263088 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.262551 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f4054b54-56fb-46d9-8302-2618330b5e9a-tmp\") pod \"tuned-fhjzk\" (UID: \"f4054b54-56fb-46d9-8302-2618330b5e9a\") " pod="openshift-cluster-node-tuning-operator/tuned-fhjzk" Jul 13 17:05:06.400172 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.268222 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxp6n\" (UniqueName: \"kubernetes.io/projected/f4054b54-56fb-46d9-8302-2618330b5e9a-kube-api-access-bxp6n\") pod \"tuned-fhjzk\" (UID: \"f4054b54-56fb-46d9-8302-2618330b5e9a\") " pod="openshift-cluster-node-tuning-operator/tuned-fhjzk" Jul 13 17:05:06.400172 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.268879 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdxvt\" (UniqueName: \"kubernetes.io/projected/6c4921ee-5d95-4ec9-b2df-e42ae98f491a-kube-api-access-gdxvt\") pod \"aws-ebs-csi-driver-node-c7g2s\" (UID: \"6c4921ee-5d95-4ec9-b2df-e42ae98f491a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7g2s" Jul 13 17:05:06.400172 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.339063 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tnc2p" Jul 13 17:05:06.400172 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.346293 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-psn5f" Jul 13 17:05:06.400172 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:06.348583 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd401c2d6_40c6_4728_a35c_1efd3f17bb9f.slice/crio-5362dfdbf600d1d093a5f989b1920274cf414f6af7c9c12134c83659af30e145 WatchSource:0}: Error finding container 5362dfdbf600d1d093a5f989b1920274cf414f6af7c9c12134c83659af30e145: Status 404 returned error can't find the container with id 5362dfdbf600d1d093a5f989b1920274cf414f6af7c9c12134c83659af30e145 Jul 13 17:05:06.400172 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.356653 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-cwgjn" Jul 13 17:05:06.400172 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:06.356986 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99b9c479_5323_44a6_bbfa_2aa301efb6af.slice/crio-e4fe110f6a694fd9a1b0e66d73d70638dab9b6bf35e34a52376bd9fb33f40367 WatchSource:0}: Error finding container e4fe110f6a694fd9a1b0e66d73d70638dab9b6bf35e34a52376bd9fb33f40367: Status 404 returned error can't find the container with id e4fe110f6a694fd9a1b0e66d73d70638dab9b6bf35e34a52376bd9fb33f40367 Jul 13 17:05:06.400172 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.360163 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xpws4" Jul 13 17:05:06.400172 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.367106 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-phcxr" Jul 13 17:05:06.400172 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.379741 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-qn7kc" Jul 13 17:05:06.400172 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.387606 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:06.400172 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.394545 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7g2s" Jul 13 17:05:06.400172 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.399324 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fhjzk" Jul 13 17:05:06.707768 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.667235 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7cb8e14a-393f-43f2-b75a-7f14756cfb2a-metrics-certs\") pod \"network-metrics-daemon-t6r98\" (UID: \"7cb8e14a-393f-43f2-b75a-7f14756cfb2a\") " pod="openshift-multus/network-metrics-daemon-t6r98" Jul 13 17:05:06.707768 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.667276 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2263bf98-5f88-41d9-b755-9f874a3d5916-original-pull-secret\") pod \"global-pull-secret-syncer-nfjpb\" (UID: \"2263bf98-5f88-41d9-b755-9f874a3d5916\") " pod="kube-system/global-pull-secret-syncer-nfjpb" Jul 13 17:05:06.707768 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:06.667299 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6hpn\" (UniqueName: \"kubernetes.io/projected/5d1dd68a-166a-4be1-812e-3e15e056c3f2-kube-api-access-z6hpn\") pod \"network-check-target-2b6wx\" (UID: \"5d1dd68a-166a-4be1-812e-3e15e056c3f2\") " pod="openshift-network-diagnostics/network-check-target-2b6wx" Jul 13 17:05:06.707768 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:06.667406 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 17:05:06.707768 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:06.667425 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 17:05:06.707768 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:06.667446 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 17:05:06.707768 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:06.667460 2569 projected.go:196] Error preparing data for projected volume kube-api-access-z6hpn for pod openshift-network-diagnostics/network-check-target-2b6wx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 17:05:06.707768 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:06.667483 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2263bf98-5f88-41d9-b755-9f874a3d5916-original-pull-secret podName:2263bf98-5f88-41d9-b755-9f874a3d5916 nodeName:}" failed. No retries permitted until 2026-07-13 17:05:07.667460514 +0000 UTC m=+4.065118915 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2263bf98-5f88-41d9-b755-9f874a3d5916-original-pull-secret") pod "global-pull-secret-syncer-nfjpb" (UID: "2263bf98-5f88-41d9-b755-9f874a3d5916") : object "kube-system"/"original-pull-secret" not registered Jul 13 17:05:06.707768 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:06.667515 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5d1dd68a-166a-4be1-812e-3e15e056c3f2-kube-api-access-z6hpn podName:5d1dd68a-166a-4be1-812e-3e15e056c3f2 nodeName:}" failed. No retries permitted until 2026-07-13 17:05:07.667499809 +0000 UTC m=+4.065158178 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-z6hpn" (UniqueName: "kubernetes.io/projected/5d1dd68a-166a-4be1-812e-3e15e056c3f2-kube-api-access-z6hpn") pod "network-check-target-2b6wx" (UID: "5d1dd68a-166a-4be1-812e-3e15e056c3f2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 17:05:06.707768 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:06.667406 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 17:05:06.707768 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:06.667588 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cb8e14a-393f-43f2-b75a-7f14756cfb2a-metrics-certs podName:7cb8e14a-393f-43f2-b75a-7f14756cfb2a nodeName:}" failed. No retries permitted until 2026-07-13 17:05:07.667550973 +0000 UTC m=+4.065209332 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7cb8e14a-393f-43f2-b75a-7f14756cfb2a-metrics-certs") pod "network-metrics-daemon-t6r98" (UID: "7cb8e14a-393f-43f2-b75a-7f14756cfb2a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 17:05:06.895919 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:06.895880 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a531b70_4a58_4752_8fa8_5ba634b542ea.slice/crio-d84a68c58fcd5908b6473d719eba7f42e9945ebf5c70dd2b19dc86d900bd86eb WatchSource:0}: Error finding container d84a68c58fcd5908b6473d719eba7f42e9945ebf5c70dd2b19dc86d900bd86eb: Status 404 returned error can't find the container with id d84a68c58fcd5908b6473d719eba7f42e9945ebf5c70dd2b19dc86d900bd86eb Jul 13 17:05:06.897070 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:06.897042 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3ec2a99_eadb_4865_ae2a_dd77aab461de.slice/crio-ab535d44036d0945af4ddba016ec5b8613cb37dad674701adc3810982c4cc2a1 WatchSource:0}: Error finding container ab535d44036d0945af4ddba016ec5b8613cb37dad674701adc3810982c4cc2a1: Status 404 returned error can't find the container with id ab535d44036d0945af4ddba016ec5b8613cb37dad674701adc3810982c4cc2a1 Jul 13 17:05:06.902953 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:06.902928 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4054b54_56fb_46d9_8302_2618330b5e9a.slice/crio-20cd9756abe1684c1354d582b87342fad50c95ed26560b62e1245325c5c898b8 WatchSource:0}: Error finding container 20cd9756abe1684c1354d582b87342fad50c95ed26560b62e1245325c5c898b8: Status 404 returned error can't find the container with id 20cd9756abe1684c1354d582b87342fad50c95ed26560b62e1245325c5c898b8 Jul 13 17:05:06.903970 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:06.903946 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7a4b481_50e3_4bb3_bcb5_6c5942814726.slice/crio-00aab0d5a2679fe20dd1e0d64c4986b0856a9238fb161f730b235dbc271561fc WatchSource:0}: Error finding container 00aab0d5a2679fe20dd1e0d64c4986b0856a9238fb161f730b235dbc271561fc: Status 404 returned error can't find the container with id 00aab0d5a2679fe20dd1e0d64c4986b0856a9238fb161f730b235dbc271561fc Jul 13 17:05:07.081985 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:07.081921 2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-12 17:00:06 +0000 UTC" deadline="2028-04-10 21:55:15.263726766 +0000 UTC" Jul 13 17:05:07.081985 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:07.081981 2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15292h50m8.181749759s" Jul 13 17:05:07.159978 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:07.159943 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tnc2p" event={"ID":"d401c2d6-40c6-4728-a35c-1efd3f17bb9f","Type":"ContainerStarted","Data":"5362dfdbf600d1d093a5f989b1920274cf414f6af7c9c12134c83659af30e145"} Jul 13 17:05:07.164341 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:07.162656 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-phcxr" event={"ID":"b3ec2a99-eadb-4865-ae2a-dd77aab461de","Type":"ContainerStarted","Data":"ab535d44036d0945af4ddba016ec5b8613cb37dad674701adc3810982c4cc2a1"} Jul 13 17:05:07.164341 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:07.163912 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" event={"ID":"6a531b70-4a58-4752-8fa8-5ba634b542ea","Type":"ContainerStarted","Data":"d84a68c58fcd5908b6473d719eba7f42e9945ebf5c70dd2b19dc86d900bd86eb"} Jul 13 17:05:07.165318 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:07.165288 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-psn5f" event={"ID":"99b9c479-5323-44a6-bbfa-2aa301efb6af","Type":"ContainerStarted","Data":"e4fe110f6a694fd9a1b0e66d73d70638dab9b6bf35e34a52376bd9fb33f40367"} Jul 13 17:05:07.166632 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:07.166576 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-qn7kc" event={"ID":"a7a4b481-50e3-4bb3-bcb5-6c5942814726","Type":"ContainerStarted","Data":"00aab0d5a2679fe20dd1e0d64c4986b0856a9238fb161f730b235dbc271561fc"} Jul 13 17:05:07.167768 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:07.167746 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fhjzk" event={"ID":"f4054b54-56fb-46d9-8302-2618330b5e9a","Type":"ContainerStarted","Data":"20cd9756abe1684c1354d582b87342fad50c95ed26560b62e1245325c5c898b8"} Jul 13 17:05:07.169442 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:07.169420 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xpws4" event={"ID":"921a5cfd-83c7-4db7-be13-53e44c5df655","Type":"ContainerStarted","Data":"907d7af95107281a6ab395cedf56a5655744a46820ab9da0582ff375b926edc8"} Jul 13 17:05:07.171229 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:07.171207 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-cwgjn" event={"ID":"252eeac8-ef53-439f-8d2f-8ef03a102104","Type":"ContainerStarted","Data":"b1833fbef01d35fa1caf82cf5d2f336e0fa10d0d4ee8b887b28422d250cf6e02"} Jul 13 17:05:07.172731 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:07.172695 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7g2s" event={"ID":"6c4921ee-5d95-4ec9-b2df-e42ae98f491a","Type":"ContainerStarted","Data":"651ae963083fc50010e213b6cedd4e31e75423db93f89c2d8d5752924dc62352"} Jul 13 17:05:07.675982 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:07.675669 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7cb8e14a-393f-43f2-b75a-7f14756cfb2a-metrics-certs\") pod \"network-metrics-daemon-t6r98\" (UID: \"7cb8e14a-393f-43f2-b75a-7f14756cfb2a\") " pod="openshift-multus/network-metrics-daemon-t6r98" Jul 13 17:05:07.676151 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:07.676090 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2263bf98-5f88-41d9-b755-9f874a3d5916-original-pull-secret\") pod \"global-pull-secret-syncer-nfjpb\" (UID: \"2263bf98-5f88-41d9-b755-9f874a3d5916\") " pod="kube-system/global-pull-secret-syncer-nfjpb" Jul 13 17:05:07.676151 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:07.676125 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6hpn\" (UniqueName: \"kubernetes.io/projected/5d1dd68a-166a-4be1-812e-3e15e056c3f2-kube-api-access-z6hpn\") pod \"network-check-target-2b6wx\" (UID: \"5d1dd68a-166a-4be1-812e-3e15e056c3f2\") " pod="openshift-network-diagnostics/network-check-target-2b6wx" Jul 13 17:05:07.676298 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:07.676279 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 17:05:07.676356 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:07.676309 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 17:05:07.676356 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:07.676322 2569 projected.go:196] Error preparing data for projected volume kube-api-access-z6hpn for pod openshift-network-diagnostics/network-check-target-2b6wx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 17:05:07.676452 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:07.676379 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5d1dd68a-166a-4be1-812e-3e15e056c3f2-kube-api-access-z6hpn podName:5d1dd68a-166a-4be1-812e-3e15e056c3f2 nodeName:}" failed. No retries permitted until 2026-07-13 17:05:09.676359311 +0000 UTC m=+6.074017670 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-z6hpn" (UniqueName: "kubernetes.io/projected/5d1dd68a-166a-4be1-812e-3e15e056c3f2-kube-api-access-z6hpn") pod "network-check-target-2b6wx" (UID: "5d1dd68a-166a-4be1-812e-3e15e056c3f2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 17:05:07.676452 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:07.676451 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 17:05:07.676583 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:07.676483 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cb8e14a-393f-43f2-b75a-7f14756cfb2a-metrics-certs podName:7cb8e14a-393f-43f2-b75a-7f14756cfb2a nodeName:}" failed. No retries permitted until 2026-07-13 17:05:09.676472078 +0000 UTC m=+6.074130437 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7cb8e14a-393f-43f2-b75a-7f14756cfb2a-metrics-certs") pod "network-metrics-daemon-t6r98" (UID: "7cb8e14a-393f-43f2-b75a-7f14756cfb2a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 17:05:07.676583 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:07.676532 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 17:05:07.676583 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:07.676559 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2263bf98-5f88-41d9-b755-9f874a3d5916-original-pull-secret podName:2263bf98-5f88-41d9-b755-9f874a3d5916 nodeName:}" failed. No retries permitted until 2026-07-13 17:05:09.67654987 +0000 UTC m=+6.074208230 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2263bf98-5f88-41d9-b755-9f874a3d5916-original-pull-secret") pod "global-pull-secret-syncer-nfjpb" (UID: "2263bf98-5f88-41d9-b755-9f874a3d5916") : object "kube-system"/"original-pull-secret" not registered Jul 13 17:05:08.082608 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:08.082555 2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-12 17:00:06 +0000 UTC" deadline="2028-02-29 05:50:25.995505436 +0000 UTC" Jul 13 17:05:08.082608 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:08.082605 2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14292h45m17.91290441s" Jul 13 17:05:08.131119 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:08.131092 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nfjpb" Jul 13 17:05:08.131262 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:08.131206 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nfjpb" podUID="2263bf98-5f88-41d9-b755-9f874a3d5916" Jul 13 17:05:08.131653 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:08.131633 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6r98" Jul 13 17:05:08.131753 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:08.131734 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6r98" podUID="7cb8e14a-393f-43f2-b75a-7f14756cfb2a" Jul 13 17:05:08.131824 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:08.131798 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2b6wx" Jul 13 17:05:08.131881 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:08.131867 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2b6wx" podUID="5d1dd68a-166a-4be1-812e-3e15e056c3f2" Jul 13 17:05:08.179308 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:08.179085 2569 generic.go:356] "Generic (PLEG): container finished" podID="d00899e62aada42f2f0bdc458167583d" containerID="a5614df01defb5871337c1d99033fa4b2913a0df95128e5c9e9879a2a9227259" exitCode=0 Jul 13 17:05:08.179308 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:08.179129 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-220.ec2.internal" event={"ID":"d00899e62aada42f2f0bdc458167583d","Type":"ContainerDied","Data":"a5614df01defb5871337c1d99033fa4b2913a0df95128e5c9e9879a2a9227259"} Jul 13 17:05:09.692510 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:09.692463 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7cb8e14a-393f-43f2-b75a-7f14756cfb2a-metrics-certs\") pod \"network-metrics-daemon-t6r98\" (UID: \"7cb8e14a-393f-43f2-b75a-7f14756cfb2a\") " pod="openshift-multus/network-metrics-daemon-t6r98" Jul 13 17:05:09.692985 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:09.692528 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2263bf98-5f88-41d9-b755-9f874a3d5916-original-pull-secret\") pod \"global-pull-secret-syncer-nfjpb\" (UID: \"2263bf98-5f88-41d9-b755-9f874a3d5916\") " pod="kube-system/global-pull-secret-syncer-nfjpb" Jul 13 17:05:09.692985 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:09.692580 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6hpn\" (UniqueName: \"kubernetes.io/projected/5d1dd68a-166a-4be1-812e-3e15e056c3f2-kube-api-access-z6hpn\") pod \"network-check-target-2b6wx\" (UID: \"5d1dd68a-166a-4be1-812e-3e15e056c3f2\") " pod="openshift-network-diagnostics/network-check-target-2b6wx" Jul 13 17:05:09.692985 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:09.692728 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 17:05:09.692985 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:09.692751 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 17:05:09.692985 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:09.692764 2569 projected.go:196] Error preparing data for projected volume kube-api-access-z6hpn for pod openshift-network-diagnostics/network-check-target-2b6wx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 17:05:09.692985 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:09.692826 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5d1dd68a-166a-4be1-812e-3e15e056c3f2-kube-api-access-z6hpn podName:5d1dd68a-166a-4be1-812e-3e15e056c3f2 nodeName:}" failed. No retries permitted until 2026-07-13 17:05:13.692807578 +0000 UTC m=+10.090465958 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-z6hpn" (UniqueName: "kubernetes.io/projected/5d1dd68a-166a-4be1-812e-3e15e056c3f2-kube-api-access-z6hpn") pod "network-check-target-2b6wx" (UID: "5d1dd68a-166a-4be1-812e-3e15e056c3f2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 17:05:09.693296 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:09.693252 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 17:05:09.693354 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:09.693306 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cb8e14a-393f-43f2-b75a-7f14756cfb2a-metrics-certs podName:7cb8e14a-393f-43f2-b75a-7f14756cfb2a nodeName:}" failed. No retries permitted until 2026-07-13 17:05:13.693290278 +0000 UTC m=+10.090948649 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7cb8e14a-393f-43f2-b75a-7f14756cfb2a-metrics-certs") pod "network-metrics-daemon-t6r98" (UID: "7cb8e14a-393f-43f2-b75a-7f14756cfb2a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 17:05:09.693419 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:09.693367 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 17:05:09.693419 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:09.693400 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2263bf98-5f88-41d9-b755-9f874a3d5916-original-pull-secret podName:2263bf98-5f88-41d9-b755-9f874a3d5916 nodeName:}" failed. No retries permitted until 2026-07-13 17:05:13.693388912 +0000 UTC m=+10.091047283 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2263bf98-5f88-41d9-b755-9f874a3d5916-original-pull-secret") pod "global-pull-secret-syncer-nfjpb" (UID: "2263bf98-5f88-41d9-b755-9f874a3d5916") : object "kube-system"/"original-pull-secret" not registered Jul 13 17:05:10.134787 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:10.134759 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6r98" Jul 13 17:05:10.134948 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:10.134877 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6r98" podUID="7cb8e14a-393f-43f2-b75a-7f14756cfb2a" Jul 13 17:05:10.135253 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:10.135233 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2b6wx" Jul 13 17:05:10.135369 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:10.135324 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2b6wx" podUID="5d1dd68a-166a-4be1-812e-3e15e056c3f2" Jul 13 17:05:10.135440 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:10.135371 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nfjpb" Jul 13 17:05:10.135489 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:10.135440 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nfjpb" podUID="2263bf98-5f88-41d9-b755-9f874a3d5916" Jul 13 17:05:12.131927 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:12.131892 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2b6wx" Jul 13 17:05:12.132341 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:12.132030 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2b6wx" podUID="5d1dd68a-166a-4be1-812e-3e15e056c3f2" Jul 13 17:05:12.132470 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:12.132445 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nfjpb" Jul 13 17:05:12.132603 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:12.132547 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nfjpb" podUID="2263bf98-5f88-41d9-b755-9f874a3d5916" Jul 13 17:05:12.132667 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:12.132617 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6r98" Jul 13 17:05:12.132757 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:12.132735 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6r98" podUID="7cb8e14a-393f-43f2-b75a-7f14756cfb2a" Jul 13 17:05:13.723907 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:13.723870 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2263bf98-5f88-41d9-b755-9f874a3d5916-original-pull-secret\") pod \"global-pull-secret-syncer-nfjpb\" (UID: \"2263bf98-5f88-41d9-b755-9f874a3d5916\") " pod="kube-system/global-pull-secret-syncer-nfjpb" Jul 13 17:05:13.724347 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:13.723922 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6hpn\" (UniqueName: \"kubernetes.io/projected/5d1dd68a-166a-4be1-812e-3e15e056c3f2-kube-api-access-z6hpn\") pod \"network-check-target-2b6wx\" (UID: \"5d1dd68a-166a-4be1-812e-3e15e056c3f2\") " pod="openshift-network-diagnostics/network-check-target-2b6wx" Jul 13 17:05:13.724347 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:13.723990 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7cb8e14a-393f-43f2-b75a-7f14756cfb2a-metrics-certs\") pod \"network-metrics-daemon-t6r98\" (UID: \"7cb8e14a-393f-43f2-b75a-7f14756cfb2a\") " pod="openshift-multus/network-metrics-daemon-t6r98" Jul 13 17:05:13.724347 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:13.724104 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 17:05:13.724347 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:13.724161 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cb8e14a-393f-43f2-b75a-7f14756cfb2a-metrics-certs podName:7cb8e14a-393f-43f2-b75a-7f14756cfb2a nodeName:}" failed. No retries permitted until 2026-07-13 17:05:21.72414247 +0000 UTC m=+18.121800833 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7cb8e14a-393f-43f2-b75a-7f14756cfb2a-metrics-certs") pod "network-metrics-daemon-t6r98" (UID: "7cb8e14a-393f-43f2-b75a-7f14756cfb2a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 17:05:13.724594 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:13.724532 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 17:05:13.724594 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:13.724593 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2263bf98-5f88-41d9-b755-9f874a3d5916-original-pull-secret podName:2263bf98-5f88-41d9-b755-9f874a3d5916 nodeName:}" failed. No retries permitted until 2026-07-13 17:05:21.724579057 +0000 UTC m=+18.122237416 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2263bf98-5f88-41d9-b755-9f874a3d5916-original-pull-secret") pod "global-pull-secret-syncer-nfjpb" (UID: "2263bf98-5f88-41d9-b755-9f874a3d5916") : object "kube-system"/"original-pull-secret" not registered Jul 13 17:05:13.724704 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:13.724671 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 17:05:13.724704 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:13.724688 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 17:05:13.724704 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:13.724699 2569 projected.go:196] Error preparing data for projected volume kube-api-access-z6hpn for pod openshift-network-diagnostics/network-check-target-2b6wx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 17:05:13.724839 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:13.724734 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5d1dd68a-166a-4be1-812e-3e15e056c3f2-kube-api-access-z6hpn podName:5d1dd68a-166a-4be1-812e-3e15e056c3f2 nodeName:}" failed. No retries permitted until 2026-07-13 17:05:21.724723605 +0000 UTC m=+18.122381965 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-z6hpn" (UniqueName: "kubernetes.io/projected/5d1dd68a-166a-4be1-812e-3e15e056c3f2-kube-api-access-z6hpn") pod "network-check-target-2b6wx" (UID: "5d1dd68a-166a-4be1-812e-3e15e056c3f2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 17:05:14.134897 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:14.134099 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2b6wx" Jul 13 17:05:14.134897 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:14.134211 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2b6wx" podUID="5d1dd68a-166a-4be1-812e-3e15e056c3f2" Jul 13 17:05:14.134897 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:14.134615 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nfjpb" Jul 13 17:05:14.134897 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:14.134712 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nfjpb" podUID="2263bf98-5f88-41d9-b755-9f874a3d5916" Jul 13 17:05:14.134897 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:14.134762 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6r98" Jul 13 17:05:14.134897 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:14.134851 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6r98" podUID="7cb8e14a-393f-43f2-b75a-7f14756cfb2a" Jul 13 17:05:16.131614 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:16.131558 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nfjpb" Jul 13 17:05:16.132000 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:16.131725 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6r98" Jul 13 17:05:16.132000 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:16.131728 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nfjpb" podUID="2263bf98-5f88-41d9-b755-9f874a3d5916" Jul 13 17:05:16.132000 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:16.131867 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6r98" podUID="7cb8e14a-393f-43f2-b75a-7f14756cfb2a" Jul 13 17:05:16.132000 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:16.131934 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2b6wx" Jul 13 17:05:16.132184 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:16.132041 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2b6wx" podUID="5d1dd68a-166a-4be1-812e-3e15e056c3f2" Jul 13 17:05:18.130935 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:18.130896 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6r98" Jul 13 17:05:18.131405 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:18.130916 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nfjpb" Jul 13 17:05:18.131405 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:18.131030 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6r98" podUID="7cb8e14a-393f-43f2-b75a-7f14756cfb2a" Jul 13 17:05:18.131405 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:18.131062 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2b6wx" Jul 13 17:05:18.131405 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:18.131159 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nfjpb" podUID="2263bf98-5f88-41d9-b755-9f874a3d5916" Jul 13 17:05:18.131405 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:18.131234 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2b6wx" podUID="5d1dd68a-166a-4be1-812e-3e15e056c3f2" Jul 13 17:05:20.131458 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:20.131418 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nfjpb" Jul 13 17:05:20.131919 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:20.131422 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6r98" Jul 13 17:05:20.131919 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:20.131552 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nfjpb" podUID="2263bf98-5f88-41d9-b755-9f874a3d5916" Jul 13 17:05:20.131919 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:20.131573 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2b6wx" Jul 13 17:05:20.131919 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:20.131675 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6r98" podUID="7cb8e14a-393f-43f2-b75a-7f14756cfb2a" Jul 13 17:05:20.131919 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:20.131768 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2b6wx" podUID="5d1dd68a-166a-4be1-812e-3e15e056c3f2" Jul 13 17:05:21.786261 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:21.786226 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7cb8e14a-393f-43f2-b75a-7f14756cfb2a-metrics-certs\") pod \"network-metrics-daemon-t6r98\" (UID: \"7cb8e14a-393f-43f2-b75a-7f14756cfb2a\") " pod="openshift-multus/network-metrics-daemon-t6r98" Jul 13 17:05:21.786753 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:21.786281 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2263bf98-5f88-41d9-b755-9f874a3d5916-original-pull-secret\") pod \"global-pull-secret-syncer-nfjpb\" (UID: \"2263bf98-5f88-41d9-b755-9f874a3d5916\") " pod="kube-system/global-pull-secret-syncer-nfjpb" Jul 13 17:05:21.786753 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:21.786308 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6hpn\" (UniqueName: \"kubernetes.io/projected/5d1dd68a-166a-4be1-812e-3e15e056c3f2-kube-api-access-z6hpn\") pod \"network-check-target-2b6wx\" (UID: \"5d1dd68a-166a-4be1-812e-3e15e056c3f2\") " pod="openshift-network-diagnostics/network-check-target-2b6wx" Jul 13 17:05:21.786753 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:21.786383 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 17:05:21.786753 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:21.786451 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cb8e14a-393f-43f2-b75a-7f14756cfb2a-metrics-certs podName:7cb8e14a-393f-43f2-b75a-7f14756cfb2a nodeName:}" failed. No retries permitted until 2026-07-13 17:05:37.786434962 +0000 UTC m=+34.184093322 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7cb8e14a-393f-43f2-b75a-7f14756cfb2a-metrics-certs") pod "network-metrics-daemon-t6r98" (UID: "7cb8e14a-393f-43f2-b75a-7f14756cfb2a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 17:05:21.786753 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:21.786452 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 17:05:21.786753 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:21.786451 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 17:05:21.786753 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:21.786508 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2263bf98-5f88-41d9-b755-9f874a3d5916-original-pull-secret podName:2263bf98-5f88-41d9-b755-9f874a3d5916 nodeName:}" failed. No retries permitted until 2026-07-13 17:05:37.786491111 +0000 UTC m=+34.184149469 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2263bf98-5f88-41d9-b755-9f874a3d5916-original-pull-secret") pod "global-pull-secret-syncer-nfjpb" (UID: "2263bf98-5f88-41d9-b755-9f874a3d5916") : object "kube-system"/"original-pull-secret" not registered Jul 13 17:05:21.786753 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:21.786526 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 17:05:21.786753 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:21.786540 2569 projected.go:196] Error preparing data for projected volume kube-api-access-z6hpn for pod openshift-network-diagnostics/network-check-target-2b6wx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 17:05:21.786753 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:21.786589 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5d1dd68a-166a-4be1-812e-3e15e056c3f2-kube-api-access-z6hpn podName:5d1dd68a-166a-4be1-812e-3e15e056c3f2 nodeName:}" failed. No retries permitted until 2026-07-13 17:05:37.786575455 +0000 UTC m=+34.184233827 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-z6hpn" (UniqueName: "kubernetes.io/projected/5d1dd68a-166a-4be1-812e-3e15e056c3f2-kube-api-access-z6hpn") pod "network-check-target-2b6wx" (UID: "5d1dd68a-166a-4be1-812e-3e15e056c3f2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 17:05:22.131121 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:22.131031 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2b6wx" Jul 13 17:05:22.131292 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:22.131043 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nfjpb" Jul 13 17:05:22.131292 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:22.131143 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2b6wx" podUID="5d1dd68a-166a-4be1-812e-3e15e056c3f2" Jul 13 17:05:22.131292 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:22.131167 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6r98" Jul 13 17:05:22.131292 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:22.131262 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nfjpb" podUID="2263bf98-5f88-41d9-b755-9f874a3d5916" Jul 13 17:05:22.131500 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:22.131348 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6r98" podUID="7cb8e14a-393f-43f2-b75a-7f14756cfb2a" Jul 13 17:05:24.132040 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:24.132004 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2b6wx" Jul 13 17:05:24.132040 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:24.132028 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6r98" Jul 13 17:05:24.132480 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:24.132112 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2b6wx" podUID="5d1dd68a-166a-4be1-812e-3e15e056c3f2" Jul 13 17:05:24.132480 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:24.132138 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nfjpb" Jul 13 17:05:24.132480 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:24.132244 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nfjpb" podUID="2263bf98-5f88-41d9-b755-9f874a3d5916" Jul 13 17:05:24.132480 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:24.132336 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6r98" podUID="7cb8e14a-393f-43f2-b75a-7f14756cfb2a" Jul 13 17:05:25.211527 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:25.211313 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-220.ec2.internal" event={"ID":"45606745bd8d4c83782613a3e901378b","Type":"ContainerStarted","Data":"e3a17032824a445158a63d4e4ce133cb4b33f06cd39c9cc80213bce463d0506c"} Jul 13 17:05:25.212870 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:25.212809 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-qn7kc" event={"ID":"a7a4b481-50e3-4bb3-bcb5-6c5942814726","Type":"ContainerStarted","Data":"bb09774f5b57a0938a5616ecd63dcf3b66f49fca7e7911443badff08930349ec"} Jul 13 17:05:25.214382 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:25.214356 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fhjzk" event={"ID":"f4054b54-56fb-46d9-8302-2618330b5e9a","Type":"ContainerStarted","Data":"dd08dc6da719ddf070ceaec5e1231cc2dacd76d3aa24b1140e6f5806152c5b15"} Jul 13 17:05:25.215904 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:25.215879 2569 generic.go:356] "Generic (PLEG): container finished" podID="921a5cfd-83c7-4db7-be13-53e44c5df655" containerID="ed1a424972a3cebd5f474e17c8a543ea5e790c92108bcc819ffde74fe5c69c12" exitCode=0 Jul 13 17:05:25.215982 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:25.215960 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xpws4" event={"ID":"921a5cfd-83c7-4db7-be13-53e44c5df655","Type":"ContainerDied","Data":"ed1a424972a3cebd5f474e17c8a543ea5e790c92108bcc819ffde74fe5c69c12"} Jul 13 17:05:25.217827 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:25.217798 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-cwgjn" event={"ID":"252eeac8-ef53-439f-8d2f-8ef03a102104","Type":"ContainerStarted","Data":"ee3bf7db7c0e7b21478e3c080602b7adb021313db5c38c9b45ca1bac61eaaafb"} Jul 13 17:05:25.219179 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:25.219157 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7g2s" event={"ID":"6c4921ee-5d95-4ec9-b2df-e42ae98f491a","Type":"ContainerStarted","Data":"08fd126a82ac0cfe29361920ef1fac312072958d04d8d2dcfb532454b1e52b4b"} Jul 13 17:05:25.221263 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:25.221056 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-220.ec2.internal" event={"ID":"d00899e62aada42f2f0bdc458167583d","Type":"ContainerStarted","Data":"10420c503672b0c583f43f515cc2fe812f8c244b0ac3c68a91afe208774cd703"} Jul 13 17:05:25.223280 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:25.223108 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-phcxr" event={"ID":"b3ec2a99-eadb-4865-ae2a-dd77aab461de","Type":"ContainerStarted","Data":"d1e0fc6c5a73a66e863ca2662ad471fa25d2214f32dce8f3732c7055c6757428"} Jul 13 17:05:25.224617 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:25.224484 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-220.ec2.internal" podStartSLOduration=20.224449841 podStartE2EDuration="20.224449841s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 17:05:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:05:25.224402141 +0000 UTC m=+21.622060521" watchObservedRunningTime="2026-07-13 17:05:25.224449841 +0000 UTC m=+21.622108220" Jul 13 17:05:25.225475 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:25.225458 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q2pnb_6a531b70-4a58-4752-8fa8-5ba634b542ea/ovn-acl-logging/0.log" Jul 13 17:05:25.225836 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:25.225820 2569 generic.go:356] "Generic (PLEG): container finished" podID="6a531b70-4a58-4752-8fa8-5ba634b542ea" containerID="f1a8b2f4bc908348c77522b76c9d84d18ea79bdb51919394dc8c3a1669581bf4" exitCode=1 Jul 13 17:05:25.225895 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:25.225882 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" event={"ID":"6a531b70-4a58-4752-8fa8-5ba634b542ea","Type":"ContainerStarted","Data":"22386bbbef2487cf7b77a3f96e25c79635480556bd06ce67adf432ca851c2ef9"} Jul 13 17:05:25.225962 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:25.225946 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" event={"ID":"6a531b70-4a58-4752-8fa8-5ba634b542ea","Type":"ContainerStarted","Data":"35fc27915660b25f2f8009be7e231eec900e541ec32a72325a581c57cca102a1"} Jul 13 17:05:25.226015 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:25.225995 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" event={"ID":"6a531b70-4a58-4752-8fa8-5ba634b542ea","Type":"ContainerDied","Data":"f1a8b2f4bc908348c77522b76c9d84d18ea79bdb51919394dc8c3a1669581bf4"} Jul 13 17:05:25.226074 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:25.226032 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" event={"ID":"6a531b70-4a58-4752-8fa8-5ba634b542ea","Type":"ContainerStarted","Data":"7813080a39e0bc26c175d077279017b27421476fe61f543f15de9d59c0ada902"} Jul 13 17:05:25.227331 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:25.227307 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-psn5f" event={"ID":"99b9c479-5323-44a6-bbfa-2aa301efb6af","Type":"ContainerStarted","Data":"64f978712cff0928e72b9ad8d1d70a5eee64845903496b3cb40035c018c5cf49"} Jul 13 17:05:25.240913 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:25.240879 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-cwgjn" podStartSLOduration=3.326077311 podStartE2EDuration="21.240866723s" totalImagesPullingTime="17.914789412s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 17:05:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:05:25.240440039 +0000 UTC m=+21.638098443" watchObservedRunningTime="2026-07-13 17:05:25.240866723 +0000 UTC m=+21.638525099" Jul 13 17:05:25.275114 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:25.275075 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-220.ec2.internal" podStartSLOduration=20.275057816 podStartE2EDuration="20.275057816s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 17:05:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:05:25.27501706 +0000 UTC m=+21.672675439" watchObservedRunningTime="2026-07-13 17:05:25.275057816 +0000 UTC m=+21.672716196" Jul 13 17:05:25.288707 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:25.288673 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-phcxr" podStartSLOduration=11.393174677 podStartE2EDuration="21.288662006s" totalImagesPullingTime="9.895487329s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:05:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:05:25.288298771 +0000 UTC m=+21.685957150" watchObservedRunningTime="2026-07-13 17:05:25.288662006 +0000 UTC m=+21.686320384" Jul 13 17:05:25.301288 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:25.301233 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-qn7kc" podStartSLOduration=11.413666352 podStartE2EDuration="21.301221487s" totalImagesPullingTime="9.887555135s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 17:05:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:05:25.300882564 +0000 UTC m=+21.698540943" watchObservedRunningTime="2026-07-13 17:05:25.301221487 +0000 UTC m=+21.698879865" Jul 13 17:05:25.316351 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:25.316306 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-fhjzk" podStartSLOduration=3.495829182 podStartE2EDuration="21.316290578s" totalImagesPullingTime="17.820461396s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 17:05:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:05:25.315813438 +0000 UTC m=+21.713471828" watchObservedRunningTime="2026-07-13 17:05:25.316290578 +0000 UTC m=+21.713948957" Jul 13 17:05:25.330296 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:25.330268 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-psn5f" podStartSLOduration=12.091675869 podStartE2EDuration="21.330259025s" totalImagesPullingTime="9.238583156s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 17:05:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:05:25.330006903 +0000 UTC m=+21.727665283" watchObservedRunningTime="2026-07-13 17:05:25.330259025 +0000 UTC m=+21.727917403" Jul 13 17:05:25.904355 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:25.904320 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-phcxr" Jul 13 17:05:25.904998 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:25.904976 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-phcxr" Jul 13 17:05:26.130771 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:26.130742 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2b6wx" Jul 13 17:05:26.130771 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:26.130752 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nfjpb" Jul 13 17:05:26.130944 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:26.130777 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6r98" Jul 13 17:05:26.130944 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:26.130860 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2b6wx" podUID="5d1dd68a-166a-4be1-812e-3e15e056c3f2" Jul 13 17:05:26.131043 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:26.130950 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nfjpb" podUID="2263bf98-5f88-41d9-b755-9f874a3d5916" Jul 13 17:05:26.131098 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:26.131076 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6r98" podUID="7cb8e14a-393f-43f2-b75a-7f14756cfb2a" Jul 13 17:05:26.231477 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:26.231449 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tnc2p" event={"ID":"d401c2d6-40c6-4728-a35c-1efd3f17bb9f","Type":"ContainerStarted","Data":"c7967fa213fe6c65f3985ac12cb12a29b54b1e5b45cb91768db712bc084ef2ac"} Jul 13 17:05:26.235035 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:26.235016 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q2pnb_6a531b70-4a58-4752-8fa8-5ba634b542ea/ovn-acl-logging/0.log" Jul 13 17:05:26.235579 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:26.235534 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" event={"ID":"6a531b70-4a58-4752-8fa8-5ba634b542ea","Type":"ContainerStarted","Data":"58a614e5a0ed56702caaf50c89d040f1a0da915d9288f3bd136ff9de417394a3"} Jul 13 17:05:26.235670 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:26.235624 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" event={"ID":"6a531b70-4a58-4752-8fa8-5ba634b542ea","Type":"ContainerStarted","Data":"b76e0a32723c0f85b99aabd0243ed8ad0059ec14f167ed6d867bb6aa356bbcae"} Jul 13 17:05:26.235878 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:26.235854 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-phcxr" Jul 13 17:05:26.236363 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:26.236340 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-phcxr" Jul 13 17:05:26.249193 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:26.249152 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-tnc2p" podStartSLOduration=3.8988867259999997 podStartE2EDuration="22.249137999s" totalImagesPullingTime="18.350251273s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 17:05:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:05:26.248967628 +0000 UTC m=+22.646626011" watchObservedRunningTime="2026-07-13 17:05:26.249137999 +0000 UTC m=+22.646796374" Jul 13 17:05:26.266819 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:26.266791 2569 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 13 17:05:27.143411 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:27.143305 2569 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-13T17:05:26.266804998Z","UUID":"278cf30c-b8c0-4bce-a90d-48ac578e1771","Handler":null,"Name":"","Endpoint":""} Jul 13 17:05:27.145030 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:27.145002 2569 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 13 17:05:27.145148 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:27.145046 2569 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 13 17:05:27.239252 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:27.239198 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7g2s" event={"ID":"6c4921ee-5d95-4ec9-b2df-e42ae98f491a","Type":"ContainerStarted","Data":"7f0421d3f6ba3032b0b5f062c41909b11cf1f927ee9db62a44ce2171140dfd5e"} Jul 13 17:05:28.131264 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:28.131070 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2b6wx" Jul 13 17:05:28.131553 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:28.131071 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nfjpb" Jul 13 17:05:28.131553 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:28.131361 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2b6wx" podUID="5d1dd68a-166a-4be1-812e-3e15e056c3f2" Jul 13 17:05:28.131553 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:28.131071 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6r98" Jul 13 17:05:28.131553 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:28.131435 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nfjpb" podUID="2263bf98-5f88-41d9-b755-9f874a3d5916" Jul 13 17:05:28.131553 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:28.131511 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6r98" podUID="7cb8e14a-393f-43f2-b75a-7f14756cfb2a" Jul 13 17:05:28.243583 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:28.243529 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7g2s" event={"ID":"6c4921ee-5d95-4ec9-b2df-e42ae98f491a","Type":"ContainerStarted","Data":"fdc47345a63b6dc97d4aa4007b5a2df870a4f1231047bd49a65fbb686dedfd90"} Jul 13 17:05:28.247098 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:28.247077 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q2pnb_6a531b70-4a58-4752-8fa8-5ba634b542ea/ovn-acl-logging/0.log" Jul 13 17:05:28.247499 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:28.247473 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" event={"ID":"6a531b70-4a58-4752-8fa8-5ba634b542ea","Type":"ContainerStarted","Data":"e996a743633519075b0548fac4718eee12602d05888ffe89e796d79b82b4e3e0"} Jul 13 17:05:28.260254 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:28.260213 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7g2s" podStartSLOduration=4.326232417 podStartE2EDuration="24.260199542s" totalImagesPullingTime="19.933967125s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 17:05:04 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:05:28.259327667 +0000 UTC m=+24.656986048" watchObservedRunningTime="2026-07-13 17:05:28.260199542 +0000 UTC m=+24.657857972" Jul 13 17:05:30.130991 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:30.130832 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6r98" Jul 13 17:05:30.131675 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:30.130837 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2b6wx" Jul 13 17:05:30.131675 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:30.131078 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6r98" podUID="7cb8e14a-393f-43f2-b75a-7f14756cfb2a" Jul 13 17:05:30.131675 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:30.130839 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nfjpb" Jul 13 17:05:30.131675 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:30.131156 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2b6wx" podUID="5d1dd68a-166a-4be1-812e-3e15e056c3f2" Jul 13 17:05:30.131675 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:30.131234 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nfjpb" podUID="2263bf98-5f88-41d9-b755-9f874a3d5916" Jul 13 17:05:30.254830 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:30.254804 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q2pnb_6a531b70-4a58-4752-8fa8-5ba634b542ea/ovn-acl-logging/0.log" Jul 13 17:05:30.255224 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:30.255197 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" event={"ID":"6a531b70-4a58-4752-8fa8-5ba634b542ea","Type":"ContainerStarted","Data":"97b2b7ad0b33cb968219b7ca8b517c402a15ff7a15e5e2373527053e797176a5"} Jul 13 17:05:30.255494 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:30.255471 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:30.255602 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:30.255589 2569 scope.go:117] "RemoveContainer" containerID="f1a8b2f4bc908348c77522b76c9d84d18ea79bdb51919394dc8c3a1669581bf4" Jul 13 17:05:30.257017 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:30.256996 2569 generic.go:356] "Generic (PLEG): container finished" podID="921a5cfd-83c7-4db7-be13-53e44c5df655" containerID="c5696fa1ac3ef631faea6564df2ac4bb59a51719346a13141a37f610291891fc" exitCode=0 Jul 13 17:05:30.257108 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:30.257040 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xpws4" event={"ID":"921a5cfd-83c7-4db7-be13-53e44c5df655","Type":"ContainerDied","Data":"c5696fa1ac3ef631faea6564df2ac4bb59a51719346a13141a37f610291891fc"} Jul 13 17:05:30.272062 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:30.272044 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:31.263373 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:31.263344 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q2pnb_6a531b70-4a58-4752-8fa8-5ba634b542ea/ovn-acl-logging/0.log" Jul 13 17:05:31.263912 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:31.263794 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" event={"ID":"6a531b70-4a58-4752-8fa8-5ba634b542ea","Type":"ContainerStarted","Data":"887a4c9ae03ee4b6d00e01b4e8fbfeb51935f43c1653144e23011991b907ae97"} Jul 13 17:05:31.263912 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:31.263810 2569 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 13 17:05:31.264034 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:31.263930 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:31.278191 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:31.278166 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:31.291533 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:31.291493 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" podStartSLOduration=9.37901547 podStartE2EDuration="27.291479228s" totalImagesPullingTime="17.912463758s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 17:05:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:05:31.290117276 +0000 UTC m=+27.687775655" watchObservedRunningTime="2026-07-13 17:05:31.291479228 +0000 UTC m=+27.689137606" Jul 13 17:05:31.762803 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:31.762764 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-nfjpb"] Jul 13 17:05:31.762945 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:31.762913 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nfjpb" Jul 13 17:05:31.763015 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:31.762996 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nfjpb" podUID="2263bf98-5f88-41d9-b755-9f874a3d5916" Jul 13 17:05:31.765965 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:31.765936 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-t6r98"] Jul 13 17:05:31.766107 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:31.766065 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6r98" Jul 13 17:05:31.766175 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:31.766154 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6r98" podUID="7cb8e14a-393f-43f2-b75a-7f14756cfb2a" Jul 13 17:05:31.766591 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:31.766533 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2b6wx"] Jul 13 17:05:31.766700 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:31.766666 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2b6wx" Jul 13 17:05:31.766757 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:31.766742 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2b6wx" podUID="5d1dd68a-166a-4be1-812e-3e15e056c3f2" Jul 13 17:05:32.237461 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:32.237432 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:05:32.267461 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:32.267432 2569 generic.go:356] "Generic (PLEG): container finished" podID="921a5cfd-83c7-4db7-be13-53e44c5df655" containerID="47d70b6dfeacdf032df59c4ba25d5d6b3966d8d269fcbfba9b5de25ea751dcf3" exitCode=0 Jul 13 17:05:32.267800 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:32.267520 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xpws4" event={"ID":"921a5cfd-83c7-4db7-be13-53e44c5df655","Type":"ContainerDied","Data":"47d70b6dfeacdf032df59c4ba25d5d6b3966d8d269fcbfba9b5de25ea751dcf3"} Jul 13 17:05:33.131257 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:33.131085 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6r98" Jul 13 17:05:33.131410 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:33.131143 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2b6wx" Jul 13 17:05:33.131410 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:33.131337 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6r98" podUID="7cb8e14a-393f-43f2-b75a-7f14756cfb2a" Jul 13 17:05:33.131517 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:33.131421 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2b6wx" podUID="5d1dd68a-166a-4be1-812e-3e15e056c3f2" Jul 13 17:05:34.131708 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:34.131680 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nfjpb" Jul 13 17:05:34.132219 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:34.131774 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nfjpb" podUID="2263bf98-5f88-41d9-b755-9f874a3d5916" Jul 13 17:05:34.274695 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:34.274663 2569 generic.go:356] "Generic (PLEG): container finished" podID="921a5cfd-83c7-4db7-be13-53e44c5df655" containerID="51f8aaaa7363c05050071b853a67d99de7778d6daf2d1179a9f5bc8d01599e71" exitCode=0 Jul 13 17:05:34.274825 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:34.274711 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xpws4" event={"ID":"921a5cfd-83c7-4db7-be13-53e44c5df655","Type":"ContainerDied","Data":"51f8aaaa7363c05050071b853a67d99de7778d6daf2d1179a9f5bc8d01599e71"} Jul 13 17:05:35.130680 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:35.130648 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6r98" Jul 13 17:05:35.130840 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:35.130660 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2b6wx" Jul 13 17:05:35.130840 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:35.130780 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6r98" podUID="7cb8e14a-393f-43f2-b75a-7f14756cfb2a" Jul 13 17:05:35.130840 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:35.130815 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2b6wx" podUID="5d1dd68a-166a-4be1-812e-3e15e056c3f2" Jul 13 17:05:36.130812 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:36.130777 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nfjpb" Jul 13 17:05:36.131466 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:36.130956 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nfjpb" podUID="2263bf98-5f88-41d9-b755-9f874a3d5916" Jul 13 17:05:37.130947 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.130913 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6r98" Jul 13 17:05:37.131375 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:37.131050 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6r98" podUID="7cb8e14a-393f-43f2-b75a-7f14756cfb2a" Jul 13 17:05:37.131375 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.131100 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2b6wx" Jul 13 17:05:37.131375 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:37.131191 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2b6wx" podUID="5d1dd68a-166a-4be1-812e-3e15e056c3f2" Jul 13 17:05:37.451552 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.451470 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-220.ec2.internal" event="NodeReady" Jul 13 17:05:37.451710 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.451635 2569 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 13 17:05:37.482558 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.482522 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5878cf7589-sgggs"] Jul 13 17:05:37.485790 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.485162 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5878cf7589-sgggs" Jul 13 17:05:37.486314 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.486289 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-ttd4d"] Jul 13 17:05:37.487500 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.487474 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 13 17:05:37.488661 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.488632 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 13 17:05:37.488779 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.488676 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-7s9dh"] Jul 13 17:05:37.488850 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.488782 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-ctbft\"" Jul 13 17:05:37.488923 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.488905 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-ttd4d" Jul 13 17:05:37.489024 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.488998 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 13 17:05:37.493265 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.493248 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 13 17:05:37.493371 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.493245 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 13 17:05:37.493435 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.493396 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-j2xl7"] Jul 13 17:05:37.493499 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.493480 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 13 17:05:37.493602 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.493400 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-w4bnh\"" Jul 13 17:05:37.493980 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.493959 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-7s9dh" Jul 13 17:05:37.494777 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.494752 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 13 17:05:37.496616 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.495973 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-fskzj"] Jul 13 17:05:37.496616 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.496124 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-j2xl7" Jul 13 17:05:37.497152 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.497127 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-4kbmv\"" Jul 13 17:05:37.497387 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.497367 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 13 17:05:37.497387 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.497384 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 13 17:05:37.497883 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.497675 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 13 17:05:37.497883 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.497786 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 13 17:05:37.498124 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.498074 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 13 17:05:37.498596 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.498578 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 13 17:05:37.498693 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.498601 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 13 17:05:37.498931 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.498910 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-hzcql"] Jul 13 17:05:37.499030 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.499001 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-r9spw\"" Jul 13 17:05:37.499087 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.499048 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 13 17:05:37.499376 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.499358 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-fskzj" Jul 13 17:05:37.499497 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.499478 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 13 17:05:37.500725 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.500706 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-lzd48"] Jul 13 17:05:37.500873 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.500826 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-hzcql" Jul 13 17:05:37.501626 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.501444 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 13 17:05:37.501626 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.501600 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 13 17:05:37.502022 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.502001 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 13 17:05:37.502106 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.502039 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 13 17:05:37.502165 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.502129 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-78f6p\"" Jul 13 17:05:37.502579 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.502544 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 13 17:05:37.502711 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.502690 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-lzd48" Jul 13 17:05:37.504671 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.504405 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 13 17:05:37.506992 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.506970 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ddbfed7e-ef66-42cf-a6e3-481f1da70624-serving-cert\") pod \"service-ca-operator-7695fc6c8c-fskzj\" (UID: \"ddbfed7e-ef66-42cf-a6e3-481f1da70624\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-fskzj" Jul 13 17:05:37.507097 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.507010 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ddbfed7e-ef66-42cf-a6e3-481f1da70624-config\") pod \"service-ca-operator-7695fc6c8c-fskzj\" (UID: \"ddbfed7e-ef66-42cf-a6e3-481f1da70624\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-fskzj" Jul 13 17:05:37.507097 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.507063 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/46bae189-2ca3-416c-bcb7-42b1283f71b8-registry-tls\") pod \"image-registry-5878cf7589-sgggs\" (UID: \"46bae189-2ca3-416c-bcb7-42b1283f71b8\") " pod="openshift-image-registry/image-registry-5878cf7589-sgggs" Jul 13 17:05:37.507202 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.507118 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6j6m\" (UniqueName: \"kubernetes.io/projected/46bae189-2ca3-416c-bcb7-42b1283f71b8-kube-api-access-c6j6m\") pod \"image-registry-5878cf7589-sgggs\" (UID: \"46bae189-2ca3-416c-bcb7-42b1283f71b8\") " pod="openshift-image-registry/image-registry-5878cf7589-sgggs" Jul 13 17:05:37.507202 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.507146 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c40e6e88-ab3f-4e3c-a0e7-2400bda333eb-trusted-ca\") pod \"console-operator-7866497494-hzcql\" (UID: \"c40e6e88-ab3f-4e3c-a0e7-2400bda333eb\") " pod="openshift-console-operator/console-operator-7866497494-hzcql" Jul 13 17:05:37.507202 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.507187 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1a2cb5a4-a8a6-4519-b001-9606305fc3a1-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-lzd48\" (UID: \"1a2cb5a4-a8a6-4519-b001-9606305fc3a1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-lzd48" Jul 13 17:05:37.507340 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.507253 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/7549ca91-bd0b-4b2f-b76c-0a7784f2fd4c-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-ttd4d\" (UID: \"7549ca91-bd0b-4b2f-b76c-0a7784f2fd4c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-ttd4d" Jul 13 17:05:37.507340 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.507283 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxxc6\" (UniqueName: \"kubernetes.io/projected/7549ca91-bd0b-4b2f-b76c-0a7784f2fd4c-kube-api-access-vxxc6\") pod \"cluster-monitoring-operator-5f7f494dd4-ttd4d\" (UID: \"7549ca91-bd0b-4b2f-b76c-0a7784f2fd4c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-ttd4d" Jul 13 17:05:37.507340 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.507311 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bd4ce4fa-6339-4a0c-8260-c6a9538543c2-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-7s9dh\" (UID: \"bd4ce4fa-6339-4a0c-8260-c6a9538543c2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-7s9dh" Jul 13 17:05:37.507500 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.507479 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/46bae189-2ca3-416c-bcb7-42b1283f71b8-image-registry-private-configuration\") pod \"image-registry-5878cf7589-sgggs\" (UID: \"46bae189-2ca3-416c-bcb7-42b1283f71b8\") " pod="openshift-image-registry/image-registry-5878cf7589-sgggs" Jul 13 17:05:37.507648 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.507532 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c40e6e88-ab3f-4e3c-a0e7-2400bda333eb-serving-cert\") pod \"console-operator-7866497494-hzcql\" (UID: \"c40e6e88-ab3f-4e3c-a0e7-2400bda333eb\") " pod="openshift-console-operator/console-operator-7866497494-hzcql" Jul 13 17:05:37.507648 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.507582 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7549ca91-bd0b-4b2f-b76c-0a7784f2fd4c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-ttd4d\" (UID: \"7549ca91-bd0b-4b2f-b76c-0a7784f2fd4c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-ttd4d" Jul 13 17:05:37.507648 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.507625 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 13 17:05:37.507874 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.507730 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7ffdcb20-b7c8-468e-9c1f-c68e9e588a93-serving-cert\") pod \"insights-operator-68d9dd94cf-j2xl7\" (UID: \"7ffdcb20-b7c8-468e-9c1f-c68e9e588a93\") " pod="openshift-insights/insights-operator-68d9dd94cf-j2xl7" Jul 13 17:05:37.507874 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.507767 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7ffdcb20-b7c8-468e-9c1f-c68e9e588a93-tmp\") pod \"insights-operator-68d9dd94cf-j2xl7\" (UID: \"7ffdcb20-b7c8-468e-9c1f-c68e9e588a93\") " pod="openshift-insights/insights-operator-68d9dd94cf-j2xl7" Jul 13 17:05:37.507874 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.507806 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ffdcb20-b7c8-468e-9c1f-c68e9e588a93-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-j2xl7\" (UID: \"7ffdcb20-b7c8-468e-9c1f-c68e9e588a93\") " pod="openshift-insights/insights-operator-68d9dd94cf-j2xl7" Jul 13 17:05:37.507874 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.507831 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rnlx\" (UniqueName: \"kubernetes.io/projected/ddbfed7e-ef66-42cf-a6e3-481f1da70624-kube-api-access-6rnlx\") pod \"service-ca-operator-7695fc6c8c-fskzj\" (UID: \"ddbfed7e-ef66-42cf-a6e3-481f1da70624\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-fskzj" Jul 13 17:05:37.507874 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.507856 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jq8m4\" (UniqueName: \"kubernetes.io/projected/1a2cb5a4-a8a6-4519-b001-9606305fc3a1-kube-api-access-jq8m4\") pod \"cluster-samples-operator-65db858894-lzd48\" (UID: \"1a2cb5a4-a8a6-4519-b001-9606305fc3a1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-lzd48" Jul 13 17:05:37.508101 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.507885 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/46bae189-2ca3-416c-bcb7-42b1283f71b8-bound-sa-token\") pod \"image-registry-5878cf7589-sgggs\" (UID: \"46bae189-2ca3-416c-bcb7-42b1283f71b8\") " pod="openshift-image-registry/image-registry-5878cf7589-sgggs" Jul 13 17:05:37.508101 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.507910 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ffdcb20-b7c8-468e-9c1f-c68e9e588a93-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-j2xl7\" (UID: \"7ffdcb20-b7c8-468e-9c1f-c68e9e588a93\") " pod="openshift-insights/insights-operator-68d9dd94cf-j2xl7" Jul 13 17:05:37.508101 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.507941 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/46bae189-2ca3-416c-bcb7-42b1283f71b8-ca-trust-extracted\") pod \"image-registry-5878cf7589-sgggs\" (UID: \"46bae189-2ca3-416c-bcb7-42b1283f71b8\") " pod="openshift-image-registry/image-registry-5878cf7589-sgggs" Jul 13 17:05:37.508101 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.507973 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/46bae189-2ca3-416c-bcb7-42b1283f71b8-registry-certificates\") pod \"image-registry-5878cf7589-sgggs\" (UID: \"46bae189-2ca3-416c-bcb7-42b1283f71b8\") " pod="openshift-image-registry/image-registry-5878cf7589-sgggs" Jul 13 17:05:37.508101 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.508001 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j25dp\" (UniqueName: \"kubernetes.io/projected/bd4ce4fa-6339-4a0c-8260-c6a9538543c2-kube-api-access-j25dp\") pod \"kube-storage-version-migrator-operator-799cb7f958-7s9dh\" (UID: \"bd4ce4fa-6339-4a0c-8260-c6a9538543c2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-7s9dh" Jul 13 17:05:37.508101 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.508035 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/46bae189-2ca3-416c-bcb7-42b1283f71b8-trusted-ca\") pod \"image-registry-5878cf7589-sgggs\" (UID: \"46bae189-2ca3-416c-bcb7-42b1283f71b8\") " pod="openshift-image-registry/image-registry-5878cf7589-sgggs" Jul 13 17:05:37.508101 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.508061 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bd4ce4fa-6339-4a0c-8260-c6a9538543c2-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-7s9dh\" (UID: \"bd4ce4fa-6339-4a0c-8260-c6a9538543c2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-7s9dh" Jul 13 17:05:37.508101 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.508087 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbtj7\" (UniqueName: \"kubernetes.io/projected/7ffdcb20-b7c8-468e-9c1f-c68e9e588a93-kube-api-access-tbtj7\") pod \"insights-operator-68d9dd94cf-j2xl7\" (UID: \"7ffdcb20-b7c8-468e-9c1f-c68e9e588a93\") " pod="openshift-insights/insights-operator-68d9dd94cf-j2xl7" Jul 13 17:05:37.508493 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.508111 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c40e6e88-ab3f-4e3c-a0e7-2400bda333eb-config\") pod \"console-operator-7866497494-hzcql\" (UID: \"c40e6e88-ab3f-4e3c-a0e7-2400bda333eb\") " pod="openshift-console-operator/console-operator-7866497494-hzcql" Jul 13 17:05:37.508493 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.508138 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/7ffdcb20-b7c8-468e-9c1f-c68e9e588a93-snapshots\") pod \"insights-operator-68d9dd94cf-j2xl7\" (UID: \"7ffdcb20-b7c8-468e-9c1f-c68e9e588a93\") " pod="openshift-insights/insights-operator-68d9dd94cf-j2xl7" Jul 13 17:05:37.508493 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.508164 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xb85\" (UniqueName: \"kubernetes.io/projected/c40e6e88-ab3f-4e3c-a0e7-2400bda333eb-kube-api-access-9xb85\") pod \"console-operator-7866497494-hzcql\" (UID: \"c40e6e88-ab3f-4e3c-a0e7-2400bda333eb\") " pod="openshift-console-operator/console-operator-7866497494-hzcql" Jul 13 17:05:37.508493 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.508193 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/46bae189-2ca3-416c-bcb7-42b1283f71b8-installation-pull-secrets\") pod \"image-registry-5878cf7589-sgggs\" (UID: \"46bae189-2ca3-416c-bcb7-42b1283f71b8\") " pod="openshift-image-registry/image-registry-5878cf7589-sgggs" Jul 13 17:05:37.508493 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.508333 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 13 17:05:37.508907 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.508707 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-425xq\"" Jul 13 17:05:37.508907 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.508902 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 13 17:05:37.509024 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.508977 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-ttd4d"] Jul 13 17:05:37.509078 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.509058 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-wsxct\"" Jul 13 17:05:37.509499 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.509307 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 13 17:05:37.509677 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.509620 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 13 17:05:37.510078 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.510056 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-7s9dh"] Jul 13 17:05:37.511627 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.511609 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 13 17:05:37.512912 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.512887 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-lzd48"] Jul 13 17:05:37.518049 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.518032 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5878cf7589-sgggs"] Jul 13 17:05:37.518584 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.518547 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 13 17:05:37.520600 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.520583 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-fskzj"] Jul 13 17:05:37.523062 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.523036 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-rhgb9"] Jul 13 17:05:37.526320 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.526301 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-969d568db-zxb2r"] Jul 13 17:05:37.526808 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.526787 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-rhgb9" Jul 13 17:05:37.528588 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.528551 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-hzcql"] Jul 13 17:05:37.528683 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.528631 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-j2xl7"] Jul 13 17:05:37.528683 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.528676 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-969d568db-zxb2r" Jul 13 17:05:37.529593 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.529552 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-b7t5w\"" Jul 13 17:05:37.529715 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.529695 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 13 17:05:37.529786 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.529759 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 13 17:05:37.530670 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.530640 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 13 17:05:37.530765 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.530714 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-wbxl8\"" Jul 13 17:05:37.530765 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.530753 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 13 17:05:37.531210 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.531188 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 13 17:05:37.531301 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.531197 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 13 17:05:37.531450 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.531413 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 13 17:05:37.531545 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.531427 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 13 17:05:37.537600 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.537555 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-9xwzd"] Jul 13 17:05:37.539762 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.539744 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-5s55t"] Jul 13 17:05:37.540018 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.539999 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9xwzd" Jul 13 17:05:37.541718 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.541697 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-rhgb9"] Jul 13 17:05:37.541804 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.541737 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-5wvjq"] Jul 13 17:05:37.542132 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.542110 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5s55t" Jul 13 17:05:37.543721 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.543701 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 13 17:05:37.543806 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.543769 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-hq4vq"] Jul 13 17:05:37.544011 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.543994 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-5wvjq" Jul 13 17:05:37.544216 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.544198 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 13 17:05:37.544623 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.544606 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 13 17:05:37.544764 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.544738 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 13 17:05:37.544890 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.544740 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 13 17:05:37.544962 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.544685 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xtcxn\"" Jul 13 17:05:37.544962 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.544798 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-ghws2\"" Jul 13 17:05:37.545777 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.545759 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 13 17:05:37.546160 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.546140 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 13 17:05:37.546638 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.546619 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-s7vl4\"" Jul 13 17:05:37.547219 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.547203 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5799c95bf4-xw7dg"] Jul 13 17:05:37.547332 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.547315 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-hq4vq" Jul 13 17:05:37.550770 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.550035 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 13 17:05:37.550770 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.550229 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 13 17:05:37.550770 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.550459 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-whr2p\"" Jul 13 17:05:37.556941 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.556923 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc7d8764f-h64lg"] Jul 13 17:05:37.558130 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.558111 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5799c95bf4-xw7dg" Jul 13 17:05:37.559623 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.559605 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b8897fcc8-fl8sl"] Jul 13 17:05:37.559765 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.559743 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc7d8764f-h64lg" Jul 13 17:05:37.562630 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.562609 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 13 17:05:37.562978 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.562958 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 13 17:05:37.563641 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.563007 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 13 17:05:37.564617 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.563141 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 13 17:05:37.564617 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.563214 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 13 17:05:37.564741 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.564618 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 13 17:05:37.564741 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.564642 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 13 17:05:37.564741 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.564624 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 13 17:05:37.565048 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.565031 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-969d568db-zxb2r"] Jul 13 17:05:37.567141 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.567107 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-hq4vq"] Jul 13 17:05:37.567284 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.567271 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b8897fcc8-fl8sl"] Jul 13 17:05:37.568073 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.565126 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b8897fcc8-fl8sl" Jul 13 17:05:37.568655 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.567683 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9xwzd"] Jul 13 17:05:37.568755 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.568702 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5s55t"] Jul 13 17:05:37.568755 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.568731 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-5wvjq"] Jul 13 17:05:37.568755 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.568749 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5799c95bf4-xw7dg"] Jul 13 17:05:37.568919 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.568768 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc7d8764f-h64lg"] Jul 13 17:05:37.570159 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.570138 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-jpbsh\"" Jul 13 17:05:37.570278 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.570259 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 13 17:05:37.608759 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.608734 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7549ca91-bd0b-4b2f-b76c-0a7784f2fd4c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-ttd4d\" (UID: \"7549ca91-bd0b-4b2f-b76c-0a7784f2fd4c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-ttd4d" Jul 13 17:05:37.608867 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.608770 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7ffdcb20-b7c8-468e-9c1f-c68e9e588a93-serving-cert\") pod \"insights-operator-68d9dd94cf-j2xl7\" (UID: \"7ffdcb20-b7c8-468e-9c1f-c68e9e588a93\") " pod="openshift-insights/insights-operator-68d9dd94cf-j2xl7" Jul 13 17:05:37.608867 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:37.608838 2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 13 17:05:37.608981 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:37.608911 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7549ca91-bd0b-4b2f-b76c-0a7784f2fd4c-cluster-monitoring-operator-tls podName:7549ca91-bd0b-4b2f-b76c-0a7784f2fd4c nodeName:}" failed. No retries permitted until 2026-07-13 17:05:38.10888976 +0000 UTC m=+34.506548136 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7549ca91-bd0b-4b2f-b76c-0a7784f2fd4c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-ttd4d" (UID: "7549ca91-bd0b-4b2f-b76c-0a7784f2fd4c") : secret "cluster-monitoring-operator-tls" not found Jul 13 17:05:37.608981 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.608906 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkgfn\" (UniqueName: \"kubernetes.io/projected/2bfc2ce7-6ebd-4197-86fa-19d9e2b385c8-kube-api-access-hkgfn\") pod \"volume-data-source-validator-78495597f6-rhgb9\" (UID: \"2bfc2ce7-6ebd-4197-86fa-19d9e2b385c8\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-rhgb9" Jul 13 17:05:37.608981 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.608949 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66mg2\" (UniqueName: \"kubernetes.io/projected/5cbb7ab6-e43d-4ef7-b9f2-c2de3288d7ca-kube-api-access-66mg2\") pod \"klusterlet-addon-workmgr-5799c95bf4-xw7dg\" (UID: \"5cbb7ab6-e43d-4ef7-b9f2-c2de3288d7ca\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5799c95bf4-xw7dg" Jul 13 17:05:37.609110 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.608979 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/de14e3bd-ed7a-4a6c-80c5-0a631a9301d2-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6b8897fcc8-fl8sl\" (UID: \"de14e3bd-ed7a-4a6c-80c5-0a631a9301d2\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b8897fcc8-fl8sl" Jul 13 17:05:37.609110 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.609006 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7ffdcb20-b7c8-468e-9c1f-c68e9e588a93-tmp\") pod \"insights-operator-68d9dd94cf-j2xl7\" (UID: \"7ffdcb20-b7c8-468e-9c1f-c68e9e588a93\") " pod="openshift-insights/insights-operator-68d9dd94cf-j2xl7" Jul 13 17:05:37.609110 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.609051 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/57b96b57-d3e5-44f3-bcca-2a077820b845-config-volume\") pod \"dns-default-9xwzd\" (UID: \"57b96b57-d3e5-44f3-bcca-2a077820b845\") " pod="openshift-dns/dns-default-9xwzd" Jul 13 17:05:37.609110 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.609093 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ffdcb20-b7c8-468e-9c1f-c68e9e588a93-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-j2xl7\" (UID: \"7ffdcb20-b7c8-468e-9c1f-c68e9e588a93\") " pod="openshift-insights/insights-operator-68d9dd94cf-j2xl7" Jul 13 17:05:37.609312 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.609149 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rnlx\" (UniqueName: \"kubernetes.io/projected/ddbfed7e-ef66-42cf-a6e3-481f1da70624-kube-api-access-6rnlx\") pod \"service-ca-operator-7695fc6c8c-fskzj\" (UID: \"ddbfed7e-ef66-42cf-a6e3-481f1da70624\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-fskzj" Jul 13 17:05:37.609312 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.609192 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jq8m4\" (UniqueName: \"kubernetes.io/projected/1a2cb5a4-a8a6-4519-b001-9606305fc3a1-kube-api-access-jq8m4\") pod \"cluster-samples-operator-65db858894-lzd48\" (UID: \"1a2cb5a4-a8a6-4519-b001-9606305fc3a1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-lzd48" Jul 13 17:05:37.609312 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.609219 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e443fc05-8996-474b-85fa-f3a28463785a-default-certificate\") pod \"router-default-969d568db-zxb2r\" (UID: \"e443fc05-8996-474b-85fa-f3a28463785a\") " pod="openshift-ingress/router-default-969d568db-zxb2r" Jul 13 17:05:37.609312 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.609250 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/46bae189-2ca3-416c-bcb7-42b1283f71b8-bound-sa-token\") pod \"image-registry-5878cf7589-sgggs\" (UID: \"46bae189-2ca3-416c-bcb7-42b1283f71b8\") " pod="openshift-image-registry/image-registry-5878cf7589-sgggs" Jul 13 17:05:37.609312 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.609276 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ffdcb20-b7c8-468e-9c1f-c68e9e588a93-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-j2xl7\" (UID: \"7ffdcb20-b7c8-468e-9c1f-c68e9e588a93\") " pod="openshift-insights/insights-operator-68d9dd94cf-j2xl7" Jul 13 17:05:37.609312 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.609301 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/e3896c87-e2a0-4309-aa5d-bcf65d9e67e9-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5fc7d8764f-h64lg\" (UID: \"e3896c87-e2a0-4309-aa5d-bcf65d9e67e9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc7d8764f-h64lg" Jul 13 17:05:37.609625 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.609334 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/46bae189-2ca3-416c-bcb7-42b1283f71b8-ca-trust-extracted\") pod \"image-registry-5878cf7589-sgggs\" (UID: \"46bae189-2ca3-416c-bcb7-42b1283f71b8\") " pod="openshift-image-registry/image-registry-5878cf7589-sgggs" Jul 13 17:05:37.609625 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.609373 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57b96b57-d3e5-44f3-bcca-2a077820b845-tmp-dir\") pod \"dns-default-9xwzd\" (UID: \"57b96b57-d3e5-44f3-bcca-2a077820b845\") " pod="openshift-dns/dns-default-9xwzd" Jul 13 17:05:37.609625 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.609374 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7ffdcb20-b7c8-468e-9c1f-c68e9e588a93-tmp\") pod \"insights-operator-68d9dd94cf-j2xl7\" (UID: \"7ffdcb20-b7c8-468e-9c1f-c68e9e588a93\") " pod="openshift-insights/insights-operator-68d9dd94cf-j2xl7" Jul 13 17:05:37.609625 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.609395 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e443fc05-8996-474b-85fa-f3a28463785a-metrics-certs\") pod \"router-default-969d568db-zxb2r\" (UID: \"e443fc05-8996-474b-85fa-f3a28463785a\") " pod="openshift-ingress/router-default-969d568db-zxb2r" Jul 13 17:05:37.609625 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.609424 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/46bae189-2ca3-416c-bcb7-42b1283f71b8-registry-certificates\") pod \"image-registry-5878cf7589-sgggs\" (UID: \"46bae189-2ca3-416c-bcb7-42b1283f71b8\") " pod="openshift-image-registry/image-registry-5878cf7589-sgggs" Jul 13 17:05:37.609625 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.609440 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j25dp\" (UniqueName: \"kubernetes.io/projected/bd4ce4fa-6339-4a0c-8260-c6a9538543c2-kube-api-access-j25dp\") pod \"kube-storage-version-migrator-operator-799cb7f958-7s9dh\" (UID: \"bd4ce4fa-6339-4a0c-8260-c6a9538543c2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-7s9dh" Jul 13 17:05:37.609625 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.609460 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/46bae189-2ca3-416c-bcb7-42b1283f71b8-trusted-ca\") pod \"image-registry-5878cf7589-sgggs\" (UID: \"46bae189-2ca3-416c-bcb7-42b1283f71b8\") " pod="openshift-image-registry/image-registry-5878cf7589-sgggs" Jul 13 17:05:37.609625 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.609484 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bd4ce4fa-6339-4a0c-8260-c6a9538543c2-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-7s9dh\" (UID: \"bd4ce4fa-6339-4a0c-8260-c6a9538543c2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-7s9dh" Jul 13 17:05:37.609625 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.609508 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbtj7\" (UniqueName: \"kubernetes.io/projected/7ffdcb20-b7c8-468e-9c1f-c68e9e588a93-kube-api-access-tbtj7\") pod \"insights-operator-68d9dd94cf-j2xl7\" (UID: \"7ffdcb20-b7c8-468e-9c1f-c68e9e588a93\") " pod="openshift-insights/insights-operator-68d9dd94cf-j2xl7" Jul 13 17:05:37.609625 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.609523 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c40e6e88-ab3f-4e3c-a0e7-2400bda333eb-config\") pod \"console-operator-7866497494-hzcql\" (UID: \"c40e6e88-ab3f-4e3c-a0e7-2400bda333eb\") " pod="openshift-console-operator/console-operator-7866497494-hzcql" Jul 13 17:05:37.609625 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.609537 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/7ffdcb20-b7c8-468e-9c1f-c68e9e588a93-snapshots\") pod \"insights-operator-68d9dd94cf-j2xl7\" (UID: \"7ffdcb20-b7c8-468e-9c1f-c68e9e588a93\") " pod="openshift-insights/insights-operator-68d9dd94cf-j2xl7" Jul 13 17:05:37.609625 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.609557 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xb85\" (UniqueName: \"kubernetes.io/projected/c40e6e88-ab3f-4e3c-a0e7-2400bda333eb-kube-api-access-9xb85\") pod \"console-operator-7866497494-hzcql\" (UID: \"c40e6e88-ab3f-4e3c-a0e7-2400bda333eb\") " pod="openshift-console-operator/console-operator-7866497494-hzcql" Jul 13 17:05:37.609625 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.609598 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/46bae189-2ca3-416c-bcb7-42b1283f71b8-installation-pull-secrets\") pod \"image-registry-5878cf7589-sgggs\" (UID: \"46bae189-2ca3-416c-bcb7-42b1283f71b8\") " pod="openshift-image-registry/image-registry-5878cf7589-sgggs" Jul 13 17:05:37.609625 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.609614 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/e3896c87-e2a0-4309-aa5d-bcf65d9e67e9-hub\") pod \"cluster-proxy-proxy-agent-5fc7d8764f-h64lg\" (UID: \"e3896c87-e2a0-4309-aa5d-bcf65d9e67e9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc7d8764f-h64lg" Jul 13 17:05:37.609625 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.609631 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gl46r\" (UniqueName: \"kubernetes.io/projected/e3896c87-e2a0-4309-aa5d-bcf65d9e67e9-kube-api-access-gl46r\") pod \"cluster-proxy-proxy-agent-5fc7d8764f-h64lg\" (UID: \"e3896c87-e2a0-4309-aa5d-bcf65d9e67e9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc7d8764f-h64lg" Jul 13 17:05:37.610285 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.609864 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fc7pz\" (UniqueName: \"kubernetes.io/projected/de14e3bd-ed7a-4a6c-80c5-0a631a9301d2-kube-api-access-fc7pz\") pod \"managed-serviceaccount-addon-agent-6b8897fcc8-fl8sl\" (UID: \"de14e3bd-ed7a-4a6c-80c5-0a631a9301d2\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b8897fcc8-fl8sl" Jul 13 17:05:37.610285 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.609903 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3896c87-e2a0-4309-aa5d-bcf65d9e67e9-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5fc7d8764f-h64lg\" (UID: \"e3896c87-e2a0-4309-aa5d-bcf65d9e67e9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc7d8764f-h64lg" Jul 13 17:05:37.610285 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.610043 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ffdcb20-b7c8-468e-9c1f-c68e9e588a93-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-j2xl7\" (UID: \"7ffdcb20-b7c8-468e-9c1f-c68e9e588a93\") " pod="openshift-insights/insights-operator-68d9dd94cf-j2xl7" Jul 13 17:05:37.610285 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.610063 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ffdcb20-b7c8-468e-9c1f-c68e9e588a93-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-j2xl7\" (UID: \"7ffdcb20-b7c8-468e-9c1f-c68e9e588a93\") " pod="openshift-insights/insights-operator-68d9dd94cf-j2xl7" Jul 13 17:05:37.610477 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.610379 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bd4ce4fa-6339-4a0c-8260-c6a9538543c2-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-7s9dh\" (UID: \"bd4ce4fa-6339-4a0c-8260-c6a9538543c2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-7s9dh" Jul 13 17:05:37.611391 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.610773 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c40e6e88-ab3f-4e3c-a0e7-2400bda333eb-config\") pod \"console-operator-7866497494-hzcql\" (UID: \"c40e6e88-ab3f-4e3c-a0e7-2400bda333eb\") " pod="openshift-console-operator/console-operator-7866497494-hzcql" Jul 13 17:05:37.611391 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.610984 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/7ffdcb20-b7c8-468e-9c1f-c68e9e588a93-snapshots\") pod \"insights-operator-68d9dd94cf-j2xl7\" (UID: \"7ffdcb20-b7c8-468e-9c1f-c68e9e588a93\") " pod="openshift-insights/insights-operator-68d9dd94cf-j2xl7" Jul 13 17:05:37.611391 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.611060 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ddbfed7e-ef66-42cf-a6e3-481f1da70624-serving-cert\") pod \"service-ca-operator-7695fc6c8c-fskzj\" (UID: \"ddbfed7e-ef66-42cf-a6e3-481f1da70624\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-fskzj" Jul 13 17:05:37.611391 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.611110 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ddbfed7e-ef66-42cf-a6e3-481f1da70624-config\") pod \"service-ca-operator-7695fc6c8c-fskzj\" (UID: \"ddbfed7e-ef66-42cf-a6e3-481f1da70624\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-fskzj" Jul 13 17:05:37.611391 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.611130 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/46bae189-2ca3-416c-bcb7-42b1283f71b8-registry-certificates\") pod \"image-registry-5878cf7589-sgggs\" (UID: \"46bae189-2ca3-416c-bcb7-42b1283f71b8\") " pod="openshift-image-registry/image-registry-5878cf7589-sgggs" Jul 13 17:05:37.611391 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.611149 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/46bae189-2ca3-416c-bcb7-42b1283f71b8-ca-trust-extracted\") pod \"image-registry-5878cf7589-sgggs\" (UID: \"46bae189-2ca3-416c-bcb7-42b1283f71b8\") " pod="openshift-image-registry/image-registry-5878cf7589-sgggs" Jul 13 17:05:37.611391 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.611158 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/46bae189-2ca3-416c-bcb7-42b1283f71b8-registry-tls\") pod \"image-registry-5878cf7589-sgggs\" (UID: \"46bae189-2ca3-416c-bcb7-42b1283f71b8\") " pod="openshift-image-registry/image-registry-5878cf7589-sgggs" Jul 13 17:05:37.611391 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.611185 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6j6m\" (UniqueName: \"kubernetes.io/projected/46bae189-2ca3-416c-bcb7-42b1283f71b8-kube-api-access-c6j6m\") pod \"image-registry-5878cf7589-sgggs\" (UID: \"46bae189-2ca3-416c-bcb7-42b1283f71b8\") " pod="openshift-image-registry/image-registry-5878cf7589-sgggs" Jul 13 17:05:37.611391 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.611210 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c40e6e88-ab3f-4e3c-a0e7-2400bda333eb-trusted-ca\") pod \"console-operator-7866497494-hzcql\" (UID: \"c40e6e88-ab3f-4e3c-a0e7-2400bda333eb\") " pod="openshift-console-operator/console-operator-7866497494-hzcql" Jul 13 17:05:37.611391 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.611235 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e3896c87-e2a0-4309-aa5d-bcf65d9e67e9-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5fc7d8764f-h64lg\" (UID: \"e3896c87-e2a0-4309-aa5d-bcf65d9e67e9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc7d8764f-h64lg" Jul 13 17:05:37.611391 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.611260 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/afb76ab4-2a66-45a7-8c98-7c3ca6e026b9-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-5wvjq\" (UID: \"afb76ab4-2a66-45a7-8c98-7c3ca6e026b9\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-5wvjq" Jul 13 17:05:37.611391 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.611285 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2f125df3-6cb8-4ec4-bd2f-24428ce6127e-cert\") pod \"ingress-canary-5s55t\" (UID: \"2f125df3-6cb8-4ec4-bd2f-24428ce6127e\") " pod="openshift-ingress-canary/ingress-canary-5s55t" Jul 13 17:05:37.611391 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.611326 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1a2cb5a4-a8a6-4519-b001-9606305fc3a1-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-lzd48\" (UID: \"1a2cb5a4-a8a6-4519-b001-9606305fc3a1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-lzd48" Jul 13 17:05:37.611391 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.611349 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5cbb7ab6-e43d-4ef7-b9f2-c2de3288d7ca-tmp\") pod \"klusterlet-addon-workmgr-5799c95bf4-xw7dg\" (UID: \"5cbb7ab6-e43d-4ef7-b9f2-c2de3288d7ca\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5799c95bf4-xw7dg" Jul 13 17:05:37.611391 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.611373 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9wvl\" (UniqueName: \"kubernetes.io/projected/e443fc05-8996-474b-85fa-f3a28463785a-kube-api-access-l9wvl\") pod \"router-default-969d568db-zxb2r\" (UID: \"e443fc05-8996-474b-85fa-f3a28463785a\") " pod="openshift-ingress/router-default-969d568db-zxb2r" Jul 13 17:05:37.612175 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.611412 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtzml\" (UniqueName: \"kubernetes.io/projected/2eb28be8-8f93-44e3-b09b-466737efa1b2-kube-api-access-wtzml\") pod \"network-check-source-869dcfcdcd-hq4vq\" (UID: \"2eb28be8-8f93-44e3-b09b-466737efa1b2\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-hq4vq" Jul 13 17:05:37.612175 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.611440 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/7549ca91-bd0b-4b2f-b76c-0a7784f2fd4c-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-ttd4d\" (UID: \"7549ca91-bd0b-4b2f-b76c-0a7784f2fd4c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-ttd4d" Jul 13 17:05:37.612175 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.611464 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/e3896c87-e2a0-4309-aa5d-bcf65d9e67e9-ca\") pod \"cluster-proxy-proxy-agent-5fc7d8764f-h64lg\" (UID: \"e3896c87-e2a0-4309-aa5d-bcf65d9e67e9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc7d8764f-h64lg" Jul 13 17:05:37.612175 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.611489 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxxc6\" (UniqueName: \"kubernetes.io/projected/7549ca91-bd0b-4b2f-b76c-0a7784f2fd4c-kube-api-access-vxxc6\") pod \"cluster-monitoring-operator-5f7f494dd4-ttd4d\" (UID: \"7549ca91-bd0b-4b2f-b76c-0a7784f2fd4c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-ttd4d" Jul 13 17:05:37.612175 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.611516 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bd4ce4fa-6339-4a0c-8260-c6a9538543c2-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-7s9dh\" (UID: \"bd4ce4fa-6339-4a0c-8260-c6a9538543c2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-7s9dh" Jul 13 17:05:37.612175 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.611544 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/57b96b57-d3e5-44f3-bcca-2a077820b845-metrics-tls\") pod \"dns-default-9xwzd\" (UID: \"57b96b57-d3e5-44f3-bcca-2a077820b845\") " pod="openshift-dns/dns-default-9xwzd" Jul 13 17:05:37.612175 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.611598 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/afb76ab4-2a66-45a7-8c98-7c3ca6e026b9-nginx-conf\") pod \"networking-console-plugin-6cf856758c-5wvjq\" (UID: \"afb76ab4-2a66-45a7-8c98-7c3ca6e026b9\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-5wvjq" Jul 13 17:05:37.612175 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.611627 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qwrw\" (UniqueName: \"kubernetes.io/projected/57b96b57-d3e5-44f3-bcca-2a077820b845-kube-api-access-7qwrw\") pod \"dns-default-9xwzd\" (UID: \"57b96b57-d3e5-44f3-bcca-2a077820b845\") " pod="openshift-dns/dns-default-9xwzd" Jul 13 17:05:37.612175 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.611651 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xs5vt\" (UniqueName: \"kubernetes.io/projected/2f125df3-6cb8-4ec4-bd2f-24428ce6127e-kube-api-access-xs5vt\") pod \"ingress-canary-5s55t\" (UID: \"2f125df3-6cb8-4ec4-bd2f-24428ce6127e\") " pod="openshift-ingress-canary/ingress-canary-5s55t" Jul 13 17:05:37.612175 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.611682 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/5cbb7ab6-e43d-4ef7-b9f2-c2de3288d7ca-klusterlet-config\") pod \"klusterlet-addon-workmgr-5799c95bf4-xw7dg\" (UID: \"5cbb7ab6-e43d-4ef7-b9f2-c2de3288d7ca\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5799c95bf4-xw7dg" Jul 13 17:05:37.612175 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.611708 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/46bae189-2ca3-416c-bcb7-42b1283f71b8-image-registry-private-configuration\") pod \"image-registry-5878cf7589-sgggs\" (UID: \"46bae189-2ca3-416c-bcb7-42b1283f71b8\") " pod="openshift-image-registry/image-registry-5878cf7589-sgggs" Jul 13 17:05:37.612175 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.611733 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c40e6e88-ab3f-4e3c-a0e7-2400bda333eb-serving-cert\") pod \"console-operator-7866497494-hzcql\" (UID: \"c40e6e88-ab3f-4e3c-a0e7-2400bda333eb\") " pod="openshift-console-operator/console-operator-7866497494-hzcql" Jul 13 17:05:37.612175 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.611758 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e443fc05-8996-474b-85fa-f3a28463785a-service-ca-bundle\") pod \"router-default-969d568db-zxb2r\" (UID: \"e443fc05-8996-474b-85fa-f3a28463785a\") " pod="openshift-ingress/router-default-969d568db-zxb2r" Jul 13 17:05:37.612175 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.611777 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/46bae189-2ca3-416c-bcb7-42b1283f71b8-trusted-ca\") pod \"image-registry-5878cf7589-sgggs\" (UID: \"46bae189-2ca3-416c-bcb7-42b1283f71b8\") " pod="openshift-image-registry/image-registry-5878cf7589-sgggs" Jul 13 17:05:37.612175 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.611786 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e443fc05-8996-474b-85fa-f3a28463785a-stats-auth\") pod \"router-default-969d568db-zxb2r\" (UID: \"e443fc05-8996-474b-85fa-f3a28463785a\") " pod="openshift-ingress/router-default-969d568db-zxb2r" Jul 13 17:05:37.612175 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:37.611939 2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 13 17:05:37.612944 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:37.612111 2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5878cf7589-sgggs: secret "image-registry-tls" not found Jul 13 17:05:37.612944 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:37.612159 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/46bae189-2ca3-416c-bcb7-42b1283f71b8-registry-tls podName:46bae189-2ca3-416c-bcb7-42b1283f71b8 nodeName:}" failed. No retries permitted until 2026-07-13 17:05:38.112143406 +0000 UTC m=+34.509801781 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/46bae189-2ca3-416c-bcb7-42b1283f71b8-registry-tls") pod "image-registry-5878cf7589-sgggs" (UID: "46bae189-2ca3-416c-bcb7-42b1283f71b8") : secret "image-registry-tls" not found Jul 13 17:05:37.613585 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.613470 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c40e6e88-ab3f-4e3c-a0e7-2400bda333eb-trusted-ca\") pod \"console-operator-7866497494-hzcql\" (UID: \"c40e6e88-ab3f-4e3c-a0e7-2400bda333eb\") " pod="openshift-console-operator/console-operator-7866497494-hzcql" Jul 13 17:05:37.613585 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.613506 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ddbfed7e-ef66-42cf-a6e3-481f1da70624-config\") pod \"service-ca-operator-7695fc6c8c-fskzj\" (UID: \"ddbfed7e-ef66-42cf-a6e3-481f1da70624\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-fskzj" Jul 13 17:05:37.614110 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:37.613771 2569 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 13 17:05:37.614110 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:37.613832 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a2cb5a4-a8a6-4519-b001-9606305fc3a1-samples-operator-tls podName:1a2cb5a4-a8a6-4519-b001-9606305fc3a1 nodeName:}" failed. No retries permitted until 2026-07-13 17:05:38.113817624 +0000 UTC m=+34.511475997 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1a2cb5a4-a8a6-4519-b001-9606305fc3a1-samples-operator-tls") pod "cluster-samples-operator-65db858894-lzd48" (UID: "1a2cb5a4-a8a6-4519-b001-9606305fc3a1") : secret "samples-operator-tls" not found Jul 13 17:05:37.614252 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.614227 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/7549ca91-bd0b-4b2f-b76c-0a7784f2fd4c-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-ttd4d\" (UID: \"7549ca91-bd0b-4b2f-b76c-0a7784f2fd4c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-ttd4d" Jul 13 17:05:37.616195 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.615654 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/46bae189-2ca3-416c-bcb7-42b1283f71b8-installation-pull-secrets\") pod \"image-registry-5878cf7589-sgggs\" (UID: \"46bae189-2ca3-416c-bcb7-42b1283f71b8\") " pod="openshift-image-registry/image-registry-5878cf7589-sgggs" Jul 13 17:05:37.616195 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.615956 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7ffdcb20-b7c8-468e-9c1f-c68e9e588a93-serving-cert\") pod \"insights-operator-68d9dd94cf-j2xl7\" (UID: \"7ffdcb20-b7c8-468e-9c1f-c68e9e588a93\") " pod="openshift-insights/insights-operator-68d9dd94cf-j2xl7" Jul 13 17:05:37.617161 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.617127 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/46bae189-2ca3-416c-bcb7-42b1283f71b8-image-registry-private-configuration\") pod \"image-registry-5878cf7589-sgggs\" (UID: \"46bae189-2ca3-416c-bcb7-42b1283f71b8\") " pod="openshift-image-registry/image-registry-5878cf7589-sgggs" Jul 13 17:05:37.618718 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.618396 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c40e6e88-ab3f-4e3c-a0e7-2400bda333eb-serving-cert\") pod \"console-operator-7866497494-hzcql\" (UID: \"c40e6e88-ab3f-4e3c-a0e7-2400bda333eb\") " pod="openshift-console-operator/console-operator-7866497494-hzcql" Jul 13 17:05:37.618800 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.618714 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ddbfed7e-ef66-42cf-a6e3-481f1da70624-serving-cert\") pod \"service-ca-operator-7695fc6c8c-fskzj\" (UID: \"ddbfed7e-ef66-42cf-a6e3-481f1da70624\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-fskzj" Jul 13 17:05:37.621189 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.621163 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rnlx\" (UniqueName: \"kubernetes.io/projected/ddbfed7e-ef66-42cf-a6e3-481f1da70624-kube-api-access-6rnlx\") pod \"service-ca-operator-7695fc6c8c-fskzj\" (UID: \"ddbfed7e-ef66-42cf-a6e3-481f1da70624\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-fskzj" Jul 13 17:05:37.621325 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.621299 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jq8m4\" (UniqueName: \"kubernetes.io/projected/1a2cb5a4-a8a6-4519-b001-9606305fc3a1-kube-api-access-jq8m4\") pod \"cluster-samples-operator-65db858894-lzd48\" (UID: \"1a2cb5a4-a8a6-4519-b001-9606305fc3a1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-lzd48" Jul 13 17:05:37.621808 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.621788 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xb85\" (UniqueName: \"kubernetes.io/projected/c40e6e88-ab3f-4e3c-a0e7-2400bda333eb-kube-api-access-9xb85\") pod \"console-operator-7866497494-hzcql\" (UID: \"c40e6e88-ab3f-4e3c-a0e7-2400bda333eb\") " pod="openshift-console-operator/console-operator-7866497494-hzcql" Jul 13 17:05:37.623172 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.623151 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/46bae189-2ca3-416c-bcb7-42b1283f71b8-bound-sa-token\") pod \"image-registry-5878cf7589-sgggs\" (UID: \"46bae189-2ca3-416c-bcb7-42b1283f71b8\") " pod="openshift-image-registry/image-registry-5878cf7589-sgggs" Jul 13 17:05:37.623898 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.623434 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6j6m\" (UniqueName: \"kubernetes.io/projected/46bae189-2ca3-416c-bcb7-42b1283f71b8-kube-api-access-c6j6m\") pod \"image-registry-5878cf7589-sgggs\" (UID: \"46bae189-2ca3-416c-bcb7-42b1283f71b8\") " pod="openshift-image-registry/image-registry-5878cf7589-sgggs" Jul 13 17:05:37.623990 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.623833 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxxc6\" (UniqueName: \"kubernetes.io/projected/7549ca91-bd0b-4b2f-b76c-0a7784f2fd4c-kube-api-access-vxxc6\") pod \"cluster-monitoring-operator-5f7f494dd4-ttd4d\" (UID: \"7549ca91-bd0b-4b2f-b76c-0a7784f2fd4c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-ttd4d" Jul 13 17:05:37.623990 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.623968 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bd4ce4fa-6339-4a0c-8260-c6a9538543c2-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-7s9dh\" (UID: \"bd4ce4fa-6339-4a0c-8260-c6a9538543c2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-7s9dh" Jul 13 17:05:37.624167 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.624145 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j25dp\" (UniqueName: \"kubernetes.io/projected/bd4ce4fa-6339-4a0c-8260-c6a9538543c2-kube-api-access-j25dp\") pod \"kube-storage-version-migrator-operator-799cb7f958-7s9dh\" (UID: \"bd4ce4fa-6339-4a0c-8260-c6a9538543c2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-7s9dh" Jul 13 17:05:37.624861 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.624809 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbtj7\" (UniqueName: \"kubernetes.io/projected/7ffdcb20-b7c8-468e-9c1f-c68e9e588a93-kube-api-access-tbtj7\") pod \"insights-operator-68d9dd94cf-j2xl7\" (UID: \"7ffdcb20-b7c8-468e-9c1f-c68e9e588a93\") " pod="openshift-insights/insights-operator-68d9dd94cf-j2xl7" Jul 13 17:05:37.695473 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.695448 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-phcxr_b3ec2a99-eadb-4865-ae2a-dd77aab461de/konnectivity-agent/0.log" limit=1024 Jul 13 17:05:37.712243 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.712155 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/e3896c87-e2a0-4309-aa5d-bcf65d9e67e9-hub\") pod \"cluster-proxy-proxy-agent-5fc7d8764f-h64lg\" (UID: \"e3896c87-e2a0-4309-aa5d-bcf65d9e67e9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc7d8764f-h64lg" Jul 13 17:05:37.712243 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.712192 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gl46r\" (UniqueName: \"kubernetes.io/projected/e3896c87-e2a0-4309-aa5d-bcf65d9e67e9-kube-api-access-gl46r\") pod \"cluster-proxy-proxy-agent-5fc7d8764f-h64lg\" (UID: \"e3896c87-e2a0-4309-aa5d-bcf65d9e67e9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc7d8764f-h64lg" Jul 13 17:05:37.712243 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.712213 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fc7pz\" (UniqueName: \"kubernetes.io/projected/de14e3bd-ed7a-4a6c-80c5-0a631a9301d2-kube-api-access-fc7pz\") pod \"managed-serviceaccount-addon-agent-6b8897fcc8-fl8sl\" (UID: \"de14e3bd-ed7a-4a6c-80c5-0a631a9301d2\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b8897fcc8-fl8sl" Jul 13 17:05:37.712447 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.712251 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3896c87-e2a0-4309-aa5d-bcf65d9e67e9-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5fc7d8764f-h64lg\" (UID: \"e3896c87-e2a0-4309-aa5d-bcf65d9e67e9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc7d8764f-h64lg" Jul 13 17:05:37.712447 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.712301 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e3896c87-e2a0-4309-aa5d-bcf65d9e67e9-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5fc7d8764f-h64lg\" (UID: \"e3896c87-e2a0-4309-aa5d-bcf65d9e67e9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc7d8764f-h64lg" Jul 13 17:05:37.712447 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.712327 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/afb76ab4-2a66-45a7-8c98-7c3ca6e026b9-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-5wvjq\" (UID: \"afb76ab4-2a66-45a7-8c98-7c3ca6e026b9\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-5wvjq" Jul 13 17:05:37.712447 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.712347 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2f125df3-6cb8-4ec4-bd2f-24428ce6127e-cert\") pod \"ingress-canary-5s55t\" (UID: \"2f125df3-6cb8-4ec4-bd2f-24428ce6127e\") " pod="openshift-ingress-canary/ingress-canary-5s55t" Jul 13 17:05:37.712447 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.712376 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5cbb7ab6-e43d-4ef7-b9f2-c2de3288d7ca-tmp\") pod \"klusterlet-addon-workmgr-5799c95bf4-xw7dg\" (UID: \"5cbb7ab6-e43d-4ef7-b9f2-c2de3288d7ca\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5799c95bf4-xw7dg" Jul 13 17:05:37.712447 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.712400 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9wvl\" (UniqueName: \"kubernetes.io/projected/e443fc05-8996-474b-85fa-f3a28463785a-kube-api-access-l9wvl\") pod \"router-default-969d568db-zxb2r\" (UID: \"e443fc05-8996-474b-85fa-f3a28463785a\") " pod="openshift-ingress/router-default-969d568db-zxb2r" Jul 13 17:05:37.712447 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.712438 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtzml\" (UniqueName: \"kubernetes.io/projected/2eb28be8-8f93-44e3-b09b-466737efa1b2-kube-api-access-wtzml\") pod \"network-check-source-869dcfcdcd-hq4vq\" (UID: \"2eb28be8-8f93-44e3-b09b-466737efa1b2\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-hq4vq" Jul 13 17:05:37.712810 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.712460 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/e3896c87-e2a0-4309-aa5d-bcf65d9e67e9-ca\") pod \"cluster-proxy-proxy-agent-5fc7d8764f-h64lg\" (UID: \"e3896c87-e2a0-4309-aa5d-bcf65d9e67e9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc7d8764f-h64lg" Jul 13 17:05:37.712810 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.712477 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/57b96b57-d3e5-44f3-bcca-2a077820b845-metrics-tls\") pod \"dns-default-9xwzd\" (UID: \"57b96b57-d3e5-44f3-bcca-2a077820b845\") " pod="openshift-dns/dns-default-9xwzd" Jul 13 17:05:37.712810 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:37.712484 2569 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 13 17:05:37.712810 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.712510 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/afb76ab4-2a66-45a7-8c98-7c3ca6e026b9-nginx-conf\") pod \"networking-console-plugin-6cf856758c-5wvjq\" (UID: \"afb76ab4-2a66-45a7-8c98-7c3ca6e026b9\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-5wvjq" Jul 13 17:05:37.712810 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:37.712546 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afb76ab4-2a66-45a7-8c98-7c3ca6e026b9-networking-console-plugin-cert podName:afb76ab4-2a66-45a7-8c98-7c3ca6e026b9 nodeName:}" failed. No retries permitted until 2026-07-13 17:05:38.2125285 +0000 UTC m=+34.610186874 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/afb76ab4-2a66-45a7-8c98-7c3ca6e026b9-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-5wvjq" (UID: "afb76ab4-2a66-45a7-8c98-7c3ca6e026b9") : secret "networking-console-plugin-cert" not found Jul 13 17:05:37.712810 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.712596 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qwrw\" (UniqueName: \"kubernetes.io/projected/57b96b57-d3e5-44f3-bcca-2a077820b845-kube-api-access-7qwrw\") pod \"dns-default-9xwzd\" (UID: \"57b96b57-d3e5-44f3-bcca-2a077820b845\") " pod="openshift-dns/dns-default-9xwzd" Jul 13 17:05:37.712810 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.712637 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xs5vt\" (UniqueName: \"kubernetes.io/projected/2f125df3-6cb8-4ec4-bd2f-24428ce6127e-kube-api-access-xs5vt\") pod \"ingress-canary-5s55t\" (UID: \"2f125df3-6cb8-4ec4-bd2f-24428ce6127e\") " pod="openshift-ingress-canary/ingress-canary-5s55t" Jul 13 17:05:37.712810 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.712666 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/5cbb7ab6-e43d-4ef7-b9f2-c2de3288d7ca-klusterlet-config\") pod \"klusterlet-addon-workmgr-5799c95bf4-xw7dg\" (UID: \"5cbb7ab6-e43d-4ef7-b9f2-c2de3288d7ca\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5799c95bf4-xw7dg" Jul 13 17:05:37.712810 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.712698 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e443fc05-8996-474b-85fa-f3a28463785a-service-ca-bundle\") pod \"router-default-969d568db-zxb2r\" (UID: \"e443fc05-8996-474b-85fa-f3a28463785a\") " pod="openshift-ingress/router-default-969d568db-zxb2r" Jul 13 17:05:37.712810 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.712730 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e443fc05-8996-474b-85fa-f3a28463785a-stats-auth\") pod \"router-default-969d568db-zxb2r\" (UID: \"e443fc05-8996-474b-85fa-f3a28463785a\") " pod="openshift-ingress/router-default-969d568db-zxb2r" Jul 13 17:05:37.712810 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.712771 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkgfn\" (UniqueName: \"kubernetes.io/projected/2bfc2ce7-6ebd-4197-86fa-19d9e2b385c8-kube-api-access-hkgfn\") pod \"volume-data-source-validator-78495597f6-rhgb9\" (UID: \"2bfc2ce7-6ebd-4197-86fa-19d9e2b385c8\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-rhgb9" Jul 13 17:05:37.712810 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.712804 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66mg2\" (UniqueName: \"kubernetes.io/projected/5cbb7ab6-e43d-4ef7-b9f2-c2de3288d7ca-kube-api-access-66mg2\") pod \"klusterlet-addon-workmgr-5799c95bf4-xw7dg\" (UID: \"5cbb7ab6-e43d-4ef7-b9f2-c2de3288d7ca\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5799c95bf4-xw7dg" Jul 13 17:05:37.713366 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.712829 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/de14e3bd-ed7a-4a6c-80c5-0a631a9301d2-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6b8897fcc8-fl8sl\" (UID: \"de14e3bd-ed7a-4a6c-80c5-0a631a9301d2\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b8897fcc8-fl8sl" Jul 13 17:05:37.713366 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.712870 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/57b96b57-d3e5-44f3-bcca-2a077820b845-config-volume\") pod \"dns-default-9xwzd\" (UID: \"57b96b57-d3e5-44f3-bcca-2a077820b845\") " pod="openshift-dns/dns-default-9xwzd" Jul 13 17:05:37.713366 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.712911 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e443fc05-8996-474b-85fa-f3a28463785a-default-certificate\") pod \"router-default-969d568db-zxb2r\" (UID: \"e443fc05-8996-474b-85fa-f3a28463785a\") " pod="openshift-ingress/router-default-969d568db-zxb2r" Jul 13 17:05:37.713366 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.712944 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/e3896c87-e2a0-4309-aa5d-bcf65d9e67e9-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5fc7d8764f-h64lg\" (UID: \"e3896c87-e2a0-4309-aa5d-bcf65d9e67e9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc7d8764f-h64lg" Jul 13 17:05:37.713366 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.712982 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57b96b57-d3e5-44f3-bcca-2a077820b845-tmp-dir\") pod \"dns-default-9xwzd\" (UID: \"57b96b57-d3e5-44f3-bcca-2a077820b845\") " pod="openshift-dns/dns-default-9xwzd" Jul 13 17:05:37.713366 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.713008 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e443fc05-8996-474b-85fa-f3a28463785a-metrics-certs\") pod \"router-default-969d568db-zxb2r\" (UID: \"e443fc05-8996-474b-85fa-f3a28463785a\") " pod="openshift-ingress/router-default-969d568db-zxb2r" Jul 13 17:05:37.713366 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.713043 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3896c87-e2a0-4309-aa5d-bcf65d9e67e9-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5fc7d8764f-h64lg\" (UID: \"e3896c87-e2a0-4309-aa5d-bcf65d9e67e9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc7d8764f-h64lg" Jul 13 17:05:37.713366 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.713156 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/afb76ab4-2a66-45a7-8c98-7c3ca6e026b9-nginx-conf\") pod \"networking-console-plugin-6cf856758c-5wvjq\" (UID: \"afb76ab4-2a66-45a7-8c98-7c3ca6e026b9\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-5wvjq" Jul 13 17:05:37.713366 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.713162 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5cbb7ab6-e43d-4ef7-b9f2-c2de3288d7ca-tmp\") pod \"klusterlet-addon-workmgr-5799c95bf4-xw7dg\" (UID: \"5cbb7ab6-e43d-4ef7-b9f2-c2de3288d7ca\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5799c95bf4-xw7dg" Jul 13 17:05:37.713749 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:37.713390 2569 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 13 17:05:37.713749 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:37.713445 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e443fc05-8996-474b-85fa-f3a28463785a-metrics-certs podName:e443fc05-8996-474b-85fa-f3a28463785a nodeName:}" failed. No retries permitted until 2026-07-13 17:05:38.213431518 +0000 UTC m=+34.611089879 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e443fc05-8996-474b-85fa-f3a28463785a-metrics-certs") pod "router-default-969d568db-zxb2r" (UID: "e443fc05-8996-474b-85fa-f3a28463785a") : secret "router-metrics-certs-default" not found Jul 13 17:05:37.717193 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.714491 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/57b96b57-d3e5-44f3-bcca-2a077820b845-config-volume\") pod \"dns-default-9xwzd\" (UID: \"57b96b57-d3e5-44f3-bcca-2a077820b845\") " pod="openshift-dns/dns-default-9xwzd" Jul 13 17:05:37.717193 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.715223 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/e3896c87-e2a0-4309-aa5d-bcf65d9e67e9-hub\") pod \"cluster-proxy-proxy-agent-5fc7d8764f-h64lg\" (UID: \"e3896c87-e2a0-4309-aa5d-bcf65d9e67e9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc7d8764f-h64lg" Jul 13 17:05:37.717193 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.715631 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57b96b57-d3e5-44f3-bcca-2a077820b845-tmp-dir\") pod \"dns-default-9xwzd\" (UID: \"57b96b57-d3e5-44f3-bcca-2a077820b845\") " pod="openshift-dns/dns-default-9xwzd" Jul 13 17:05:37.717193 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:37.715816 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e443fc05-8996-474b-85fa-f3a28463785a-service-ca-bundle podName:e443fc05-8996-474b-85fa-f3a28463785a nodeName:}" failed. No retries permitted until 2026-07-13 17:05:38.215791285 +0000 UTC m=+34.613449657 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e443fc05-8996-474b-85fa-f3a28463785a-service-ca-bundle") pod "router-default-969d568db-zxb2r" (UID: "e443fc05-8996-474b-85fa-f3a28463785a") : configmap references non-existent config key: service-ca.crt Jul 13 17:05:37.717193 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.716769 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2f125df3-6cb8-4ec4-bd2f-24428ce6127e-cert\") pod \"ingress-canary-5s55t\" (UID: \"2f125df3-6cb8-4ec4-bd2f-24428ce6127e\") " pod="openshift-ingress-canary/ingress-canary-5s55t" Jul 13 17:05:37.717496 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.717267 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e3896c87-e2a0-4309-aa5d-bcf65d9e67e9-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5fc7d8764f-h64lg\" (UID: \"e3896c87-e2a0-4309-aa5d-bcf65d9e67e9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc7d8764f-h64lg" Jul 13 17:05:37.717496 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:37.717387 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 17:05:37.717496 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:37.717461 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57b96b57-d3e5-44f3-bcca-2a077820b845-metrics-tls podName:57b96b57-d3e5-44f3-bcca-2a077820b845 nodeName:}" failed. No retries permitted until 2026-07-13 17:05:38.217444826 +0000 UTC m=+34.615103182 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/57b96b57-d3e5-44f3-bcca-2a077820b845-metrics-tls") pod "dns-default-9xwzd" (UID: "57b96b57-d3e5-44f3-bcca-2a077820b845") : secret "dns-default-metrics-tls" not found Jul 13 17:05:37.719794 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.719769 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e443fc05-8996-474b-85fa-f3a28463785a-default-certificate\") pod \"router-default-969d568db-zxb2r\" (UID: \"e443fc05-8996-474b-85fa-f3a28463785a\") " pod="openshift-ingress/router-default-969d568db-zxb2r" Jul 13 17:05:37.721466 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.720480 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/5cbb7ab6-e43d-4ef7-b9f2-c2de3288d7ca-klusterlet-config\") pod \"klusterlet-addon-workmgr-5799c95bf4-xw7dg\" (UID: \"5cbb7ab6-e43d-4ef7-b9f2-c2de3288d7ca\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5799c95bf4-xw7dg" Jul 13 17:05:37.721466 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.720993 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e443fc05-8996-474b-85fa-f3a28463785a-stats-auth\") pod \"router-default-969d568db-zxb2r\" (UID: \"e443fc05-8996-474b-85fa-f3a28463785a\") " pod="openshift-ingress/router-default-969d568db-zxb2r" Jul 13 17:05:37.721466 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.721167 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/de14e3bd-ed7a-4a6c-80c5-0a631a9301d2-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6b8897fcc8-fl8sl\" (UID: \"de14e3bd-ed7a-4a6c-80c5-0a631a9301d2\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b8897fcc8-fl8sl" Jul 13 17:05:37.721686 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.721547 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/e3896c87-e2a0-4309-aa5d-bcf65d9e67e9-ca\") pod \"cluster-proxy-proxy-agent-5fc7d8764f-h64lg\" (UID: \"e3896c87-e2a0-4309-aa5d-bcf65d9e67e9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc7d8764f-h64lg" Jul 13 17:05:37.726611 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.722880 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9wvl\" (UniqueName: \"kubernetes.io/projected/e443fc05-8996-474b-85fa-f3a28463785a-kube-api-access-l9wvl\") pod \"router-default-969d568db-zxb2r\" (UID: \"e443fc05-8996-474b-85fa-f3a28463785a\") " pod="openshift-ingress/router-default-969d568db-zxb2r" Jul 13 17:05:37.726611 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.724449 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtzml\" (UniqueName: \"kubernetes.io/projected/2eb28be8-8f93-44e3-b09b-466737efa1b2-kube-api-access-wtzml\") pod \"network-check-source-869dcfcdcd-hq4vq\" (UID: \"2eb28be8-8f93-44e3-b09b-466737efa1b2\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-hq4vq" Jul 13 17:05:37.729057 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.729001 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xs5vt\" (UniqueName: \"kubernetes.io/projected/2f125df3-6cb8-4ec4-bd2f-24428ce6127e-kube-api-access-xs5vt\") pod \"ingress-canary-5s55t\" (UID: \"2f125df3-6cb8-4ec4-bd2f-24428ce6127e\") " pod="openshift-ingress-canary/ingress-canary-5s55t" Jul 13 17:05:37.729984 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.729937 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qwrw\" (UniqueName: \"kubernetes.io/projected/57b96b57-d3e5-44f3-bcca-2a077820b845-kube-api-access-7qwrw\") pod \"dns-default-9xwzd\" (UID: \"57b96b57-d3e5-44f3-bcca-2a077820b845\") " pod="openshift-dns/dns-default-9xwzd" Jul 13 17:05:37.730613 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.730057 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/e3896c87-e2a0-4309-aa5d-bcf65d9e67e9-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5fc7d8764f-h64lg\" (UID: \"e3896c87-e2a0-4309-aa5d-bcf65d9e67e9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc7d8764f-h64lg" Jul 13 17:05:37.730613 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.730216 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fc7pz\" (UniqueName: \"kubernetes.io/projected/de14e3bd-ed7a-4a6c-80c5-0a631a9301d2-kube-api-access-fc7pz\") pod \"managed-serviceaccount-addon-agent-6b8897fcc8-fl8sl\" (UID: \"de14e3bd-ed7a-4a6c-80c5-0a631a9301d2\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b8897fcc8-fl8sl" Jul 13 17:05:37.730613 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.730579 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkgfn\" (UniqueName: \"kubernetes.io/projected/2bfc2ce7-6ebd-4197-86fa-19d9e2b385c8-kube-api-access-hkgfn\") pod \"volume-data-source-validator-78495597f6-rhgb9\" (UID: \"2bfc2ce7-6ebd-4197-86fa-19d9e2b385c8\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-rhgb9" Jul 13 17:05:37.731503 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.731482 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gl46r\" (UniqueName: \"kubernetes.io/projected/e3896c87-e2a0-4309-aa5d-bcf65d9e67e9-kube-api-access-gl46r\") pod \"cluster-proxy-proxy-agent-5fc7d8764f-h64lg\" (UID: \"e3896c87-e2a0-4309-aa5d-bcf65d9e67e9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc7d8764f-h64lg" Jul 13 17:05:37.738768 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.738745 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66mg2\" (UniqueName: \"kubernetes.io/projected/5cbb7ab6-e43d-4ef7-b9f2-c2de3288d7ca-kube-api-access-66mg2\") pod \"klusterlet-addon-workmgr-5799c95bf4-xw7dg\" (UID: \"5cbb7ab6-e43d-4ef7-b9f2-c2de3288d7ca\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5799c95bf4-xw7dg" Jul 13 17:05:37.790724 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.790689 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-phcxr_b3ec2a99-eadb-4865-ae2a-dd77aab461de/konnectivity-agent/0.log" limit=1024 Jul 13 17:05:37.814211 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.814179 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7cb8e14a-393f-43f2-b75a-7f14756cfb2a-metrics-certs\") pod \"network-metrics-daemon-t6r98\" (UID: \"7cb8e14a-393f-43f2-b75a-7f14756cfb2a\") " pod="openshift-multus/network-metrics-daemon-t6r98" Jul 13 17:05:37.814340 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.814248 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2263bf98-5f88-41d9-b755-9f874a3d5916-original-pull-secret\") pod \"global-pull-secret-syncer-nfjpb\" (UID: \"2263bf98-5f88-41d9-b755-9f874a3d5916\") " pod="kube-system/global-pull-secret-syncer-nfjpb" Jul 13 17:05:37.814399 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:37.814346 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 17:05:37.814448 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.814393 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6hpn\" (UniqueName: \"kubernetes.io/projected/5d1dd68a-166a-4be1-812e-3e15e056c3f2-kube-api-access-z6hpn\") pod \"network-check-target-2b6wx\" (UID: \"5d1dd68a-166a-4be1-812e-3e15e056c3f2\") " pod="openshift-network-diagnostics/network-check-target-2b6wx" Jul 13 17:05:37.814448 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:37.814411 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2263bf98-5f88-41d9-b755-9f874a3d5916-original-pull-secret podName:2263bf98-5f88-41d9-b755-9f874a3d5916 nodeName:}" failed. No retries permitted until 2026-07-13 17:06:09.814396405 +0000 UTC m=+66.212054761 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2263bf98-5f88-41d9-b755-9f874a3d5916-original-pull-secret") pod "global-pull-secret-syncer-nfjpb" (UID: "2263bf98-5f88-41d9-b755-9f874a3d5916") : object "kube-system"/"original-pull-secret" not registered Jul 13 17:05:37.814555 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:37.814348 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 17:05:37.814555 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:37.814530 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cb8e14a-393f-43f2-b75a-7f14756cfb2a-metrics-certs podName:7cb8e14a-393f-43f2-b75a-7f14756cfb2a nodeName:}" failed. No retries permitted until 2026-07-13 17:06:09.814514326 +0000 UTC m=+66.212172686 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7cb8e14a-393f-43f2-b75a-7f14756cfb2a-metrics-certs") pod "network-metrics-daemon-t6r98" (UID: "7cb8e14a-393f-43f2-b75a-7f14756cfb2a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 17:05:37.816992 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.816973 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6hpn\" (UniqueName: \"kubernetes.io/projected/5d1dd68a-166a-4be1-812e-3e15e056c3f2-kube-api-access-z6hpn\") pod \"network-check-target-2b6wx\" (UID: \"5d1dd68a-166a-4be1-812e-3e15e056c3f2\") " pod="openshift-network-diagnostics/network-check-target-2b6wx" Jul 13 17:05:37.825176 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.825159 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-7s9dh" Jul 13 17:05:37.831694 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.831673 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-j2xl7" Jul 13 17:05:37.844238 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.844215 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-fskzj" Jul 13 17:05:37.852586 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.852556 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-hzcql" Jul 13 17:05:37.872637 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.872617 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-rhgb9" Jul 13 17:05:37.893577 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.893544 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5s55t" Jul 13 17:05:37.907293 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.907274 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-hq4vq" Jul 13 17:05:37.916465 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.916436 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5799c95bf4-xw7dg" Jul 13 17:05:37.956069 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.956047 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc7d8764f-h64lg" Jul 13 17:05:37.962085 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:37.962068 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b8897fcc8-fl8sl" Jul 13 17:05:38.117242 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:38.117201 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7549ca91-bd0b-4b2f-b76c-0a7784f2fd4c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-ttd4d\" (UID: \"7549ca91-bd0b-4b2f-b76c-0a7784f2fd4c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-ttd4d" Jul 13 17:05:38.117413 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:38.117329 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/46bae189-2ca3-416c-bcb7-42b1283f71b8-registry-tls\") pod \"image-registry-5878cf7589-sgggs\" (UID: \"46bae189-2ca3-416c-bcb7-42b1283f71b8\") " pod="openshift-image-registry/image-registry-5878cf7589-sgggs" Jul 13 17:05:38.117413 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:38.117364 2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 13 17:05:38.117413 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:38.117386 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1a2cb5a4-a8a6-4519-b001-9606305fc3a1-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-lzd48\" (UID: \"1a2cb5a4-a8a6-4519-b001-9606305fc3a1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-lzd48" Jul 13 17:05:38.117601 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:38.117438 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7549ca91-bd0b-4b2f-b76c-0a7784f2fd4c-cluster-monitoring-operator-tls podName:7549ca91-bd0b-4b2f-b76c-0a7784f2fd4c nodeName:}" failed. No retries permitted until 2026-07-13 17:05:39.117417517 +0000 UTC m=+35.515075873 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7549ca91-bd0b-4b2f-b76c-0a7784f2fd4c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-ttd4d" (UID: "7549ca91-bd0b-4b2f-b76c-0a7784f2fd4c") : secret "cluster-monitoring-operator-tls" not found Jul 13 17:05:38.117601 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:38.117483 2569 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 13 17:05:38.117601 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:38.117489 2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 13 17:05:38.117601 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:38.117509 2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5878cf7589-sgggs: secret "image-registry-tls" not found Jul 13 17:05:38.117601 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:38.117540 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a2cb5a4-a8a6-4519-b001-9606305fc3a1-samples-operator-tls podName:1a2cb5a4-a8a6-4519-b001-9606305fc3a1 nodeName:}" failed. No retries permitted until 2026-07-13 17:05:39.117522514 +0000 UTC m=+35.515180879 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1a2cb5a4-a8a6-4519-b001-9606305fc3a1-samples-operator-tls") pod "cluster-samples-operator-65db858894-lzd48" (UID: "1a2cb5a4-a8a6-4519-b001-9606305fc3a1") : secret "samples-operator-tls" not found Jul 13 17:05:38.117601 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:38.117557 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/46bae189-2ca3-416c-bcb7-42b1283f71b8-registry-tls podName:46bae189-2ca3-416c-bcb7-42b1283f71b8 nodeName:}" failed. No retries permitted until 2026-07-13 17:05:39.11754737 +0000 UTC m=+35.515205732 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/46bae189-2ca3-416c-bcb7-42b1283f71b8-registry-tls") pod "image-registry-5878cf7589-sgggs" (UID: "46bae189-2ca3-416c-bcb7-42b1283f71b8") : secret "image-registry-tls" not found Jul 13 17:05:38.131550 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:38.131515 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nfjpb" Jul 13 17:05:38.133648 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:38.133615 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 13 17:05:38.218690 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:38.218610 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e443fc05-8996-474b-85fa-f3a28463785a-metrics-certs\") pod \"router-default-969d568db-zxb2r\" (UID: \"e443fc05-8996-474b-85fa-f3a28463785a\") " pod="openshift-ingress/router-default-969d568db-zxb2r" Jul 13 17:05:38.218831 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:38.218721 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/afb76ab4-2a66-45a7-8c98-7c3ca6e026b9-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-5wvjq\" (UID: \"afb76ab4-2a66-45a7-8c98-7c3ca6e026b9\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-5wvjq" Jul 13 17:05:38.218831 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:38.218762 2569 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 13 17:05:38.218831 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:38.218796 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/57b96b57-d3e5-44f3-bcca-2a077820b845-metrics-tls\") pod \"dns-default-9xwzd\" (UID: \"57b96b57-d3e5-44f3-bcca-2a077820b845\") " pod="openshift-dns/dns-default-9xwzd" Jul 13 17:05:38.218831 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:38.218831 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e443fc05-8996-474b-85fa-f3a28463785a-metrics-certs podName:e443fc05-8996-474b-85fa-f3a28463785a nodeName:}" failed. No retries permitted until 2026-07-13 17:05:39.218809339 +0000 UTC m=+35.616467697 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e443fc05-8996-474b-85fa-f3a28463785a-metrics-certs") pod "router-default-969d568db-zxb2r" (UID: "e443fc05-8996-474b-85fa-f3a28463785a") : secret "router-metrics-certs-default" not found Jul 13 17:05:38.219026 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:38.218832 2569 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 13 17:05:38.219026 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:38.218879 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 17:05:38.219026 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:38.218886 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e443fc05-8996-474b-85fa-f3a28463785a-service-ca-bundle\") pod \"router-default-969d568db-zxb2r\" (UID: \"e443fc05-8996-474b-85fa-f3a28463785a\") " pod="openshift-ingress/router-default-969d568db-zxb2r" Jul 13 17:05:38.219026 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:38.218924 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57b96b57-d3e5-44f3-bcca-2a077820b845-metrics-tls podName:57b96b57-d3e5-44f3-bcca-2a077820b845 nodeName:}" failed. No retries permitted until 2026-07-13 17:05:39.218908774 +0000 UTC m=+35.616567134 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/57b96b57-d3e5-44f3-bcca-2a077820b845-metrics-tls") pod "dns-default-9xwzd" (UID: "57b96b57-d3e5-44f3-bcca-2a077820b845") : secret "dns-default-metrics-tls" not found Jul 13 17:05:38.219026 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:38.218999 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afb76ab4-2a66-45a7-8c98-7c3ca6e026b9-networking-console-plugin-cert podName:afb76ab4-2a66-45a7-8c98-7c3ca6e026b9 nodeName:}" failed. No retries permitted until 2026-07-13 17:05:39.218988681 +0000 UTC m=+35.616647043 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/afb76ab4-2a66-45a7-8c98-7c3ca6e026b9-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-5wvjq" (UID: "afb76ab4-2a66-45a7-8c98-7c3ca6e026b9") : secret "networking-console-plugin-cert" not found Jul 13 17:05:38.219026 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:38.219015 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e443fc05-8996-474b-85fa-f3a28463785a-service-ca-bundle podName:e443fc05-8996-474b-85fa-f3a28463785a nodeName:}" failed. No retries permitted until 2026-07-13 17:05:39.219006504 +0000 UTC m=+35.616664863 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e443fc05-8996-474b-85fa-f3a28463785a-service-ca-bundle") pod "router-default-969d568db-zxb2r" (UID: "e443fc05-8996-474b-85fa-f3a28463785a") : configmap references non-existent config key: service-ca.crt Jul 13 17:05:38.955160 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:38.955129 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5799c95bf4-xw7dg"] Jul 13 17:05:38.960853 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:38.960773 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-j2xl7"] Jul 13 17:05:38.964695 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:38.964667 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-7s9dh"] Jul 13 17:05:38.970191 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:38.970166 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-rhgb9"] Jul 13 17:05:38.971392 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:38.971371 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-hq4vq"] Jul 13 17:05:38.973738 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:38.973708 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5s55t"] Jul 13 17:05:38.974503 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:38.974482 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc7d8764f-h64lg"] Jul 13 17:05:38.983915 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:38.983897 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b8897fcc8-fl8sl"] Jul 13 17:05:38.987406 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:38.987362 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-hzcql"] Jul 13 17:05:38.988209 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:38.988189 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-fskzj"] Jul 13 17:05:39.013644 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:39.013277 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2eb28be8_8f93_44e3_b09b_466737efa1b2.slice/crio-caf0f9eb79e27b754b8a6bd3bd8f2c424c94e964a82e2d779a87e7236be1a6c9 WatchSource:0}: Error finding container caf0f9eb79e27b754b8a6bd3bd8f2c424c94e964a82e2d779a87e7236be1a6c9: Status 404 returned error can't find the container with id caf0f9eb79e27b754b8a6bd3bd8f2c424c94e964a82e2d779a87e7236be1a6c9 Jul 13 17:05:39.013761 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:39.013639 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f125df3_6cb8_4ec4_bd2f_24428ce6127e.slice/crio-9f3e5d3fa268bfb3aa0e5ac4d85a794135580b861d15990638b3fe66ec2fe315 WatchSource:0}: Error finding container 9f3e5d3fa268bfb3aa0e5ac4d85a794135580b861d15990638b3fe66ec2fe315: Status 404 returned error can't find the container with id 9f3e5d3fa268bfb3aa0e5ac4d85a794135580b861d15990638b3fe66ec2fe315 Jul 13 17:05:39.014667 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:39.014119 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bfc2ce7_6ebd_4197_86fa_19d9e2b385c8.slice/crio-82e10cae1901baccbdf2b603800dc325ceac8aa983e60eefc029cf76970ddb69 WatchSource:0}: Error finding container 82e10cae1901baccbdf2b603800dc325ceac8aa983e60eefc029cf76970ddb69: Status 404 returned error can't find the container with id 82e10cae1901baccbdf2b603800dc325ceac8aa983e60eefc029cf76970ddb69 Jul 13 17:05:39.130888 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:39.130822 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2b6wx" Jul 13 17:05:39.131002 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:39.130983 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6r98" Jul 13 17:05:39.135476 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:39.135453 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-wvvhh\"" Jul 13 17:05:39.135476 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:39.135464 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/46bae189-2ca3-416c-bcb7-42b1283f71b8-registry-tls\") pod \"image-registry-5878cf7589-sgggs\" (UID: \"46bae189-2ca3-416c-bcb7-42b1283f71b8\") " pod="openshift-image-registry/image-registry-5878cf7589-sgggs" Jul 13 17:05:39.135871 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:39.135524 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1a2cb5a4-a8a6-4519-b001-9606305fc3a1-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-lzd48\" (UID: \"1a2cb5a4-a8a6-4519-b001-9606305fc3a1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-lzd48" Jul 13 17:05:39.135871 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:39.135530 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 13 17:05:39.135871 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:39.135586 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5r2pv\"" Jul 13 17:05:39.135871 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:39.135630 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7549ca91-bd0b-4b2f-b76c-0a7784f2fd4c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-ttd4d\" (UID: \"7549ca91-bd0b-4b2f-b76c-0a7784f2fd4c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-ttd4d" Jul 13 17:05:39.135871 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:39.135667 2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 13 17:05:39.135871 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:39.135685 2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5878cf7589-sgggs: secret "image-registry-tls" not found Jul 13 17:05:39.135871 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:39.135700 2569 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 13 17:05:39.135871 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:39.135727 2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 13 17:05:39.135871 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:39.135739 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/46bae189-2ca3-416c-bcb7-42b1283f71b8-registry-tls podName:46bae189-2ca3-416c-bcb7-42b1283f71b8 nodeName:}" failed. No retries permitted until 2026-07-13 17:05:41.13572098 +0000 UTC m=+37.533379337 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/46bae189-2ca3-416c-bcb7-42b1283f71b8-registry-tls") pod "image-registry-5878cf7589-sgggs" (UID: "46bae189-2ca3-416c-bcb7-42b1283f71b8") : secret "image-registry-tls" not found Jul 13 17:05:39.135871 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:39.135756 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a2cb5a4-a8a6-4519-b001-9606305fc3a1-samples-operator-tls podName:1a2cb5a4-a8a6-4519-b001-9606305fc3a1 nodeName:}" failed. No retries permitted until 2026-07-13 17:05:41.135747507 +0000 UTC m=+37.533405872 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1a2cb5a4-a8a6-4519-b001-9606305fc3a1-samples-operator-tls") pod "cluster-samples-operator-65db858894-lzd48" (UID: "1a2cb5a4-a8a6-4519-b001-9606305fc3a1") : secret "samples-operator-tls" not found Jul 13 17:05:39.135871 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:39.135861 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7549ca91-bd0b-4b2f-b76c-0a7784f2fd4c-cluster-monitoring-operator-tls podName:7549ca91-bd0b-4b2f-b76c-0a7784f2fd4c nodeName:}" failed. No retries permitted until 2026-07-13 17:05:41.135845391 +0000 UTC m=+37.533503751 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7549ca91-bd0b-4b2f-b76c-0a7784f2fd4c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-ttd4d" (UID: "7549ca91-bd0b-4b2f-b76c-0a7784f2fd4c") : secret "cluster-monitoring-operator-tls" not found Jul 13 17:05:39.143699 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:39.143683 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2b6wx" Jul 13 17:05:39.236255 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:39.236227 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e443fc05-8996-474b-85fa-f3a28463785a-metrics-certs\") pod \"router-default-969d568db-zxb2r\" (UID: \"e443fc05-8996-474b-85fa-f3a28463785a\") " pod="openshift-ingress/router-default-969d568db-zxb2r" Jul 13 17:05:39.236366 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:39.236294 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/afb76ab4-2a66-45a7-8c98-7c3ca6e026b9-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-5wvjq\" (UID: \"afb76ab4-2a66-45a7-8c98-7c3ca6e026b9\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-5wvjq" Jul 13 17:05:39.236366 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:39.236349 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/57b96b57-d3e5-44f3-bcca-2a077820b845-metrics-tls\") pod \"dns-default-9xwzd\" (UID: \"57b96b57-d3e5-44f3-bcca-2a077820b845\") " pod="openshift-dns/dns-default-9xwzd" Jul 13 17:05:39.236463 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:39.236385 2569 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 13 17:05:39.236463 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:39.236420 2569 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 13 17:05:39.236546 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:39.236396 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e443fc05-8996-474b-85fa-f3a28463785a-service-ca-bundle\") pod \"router-default-969d568db-zxb2r\" (UID: \"e443fc05-8996-474b-85fa-f3a28463785a\") " pod="openshift-ingress/router-default-969d568db-zxb2r" Jul 13 17:05:39.236546 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:39.236488 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afb76ab4-2a66-45a7-8c98-7c3ca6e026b9-networking-console-plugin-cert podName:afb76ab4-2a66-45a7-8c98-7c3ca6e026b9 nodeName:}" failed. No retries permitted until 2026-07-13 17:05:41.236458718 +0000 UTC m=+37.634117088 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/afb76ab4-2a66-45a7-8c98-7c3ca6e026b9-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-5wvjq" (UID: "afb76ab4-2a66-45a7-8c98-7c3ca6e026b9") : secret "networking-console-plugin-cert" not found Jul 13 17:05:39.236671 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:39.236624 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 17:05:39.236761 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:39.236742 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e443fc05-8996-474b-85fa-f3a28463785a-metrics-certs podName:e443fc05-8996-474b-85fa-f3a28463785a nodeName:}" failed. No retries permitted until 2026-07-13 17:05:41.236724095 +0000 UTC m=+37.634382456 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e443fc05-8996-474b-85fa-f3a28463785a-metrics-certs") pod "router-default-969d568db-zxb2r" (UID: "e443fc05-8996-474b-85fa-f3a28463785a") : secret "router-metrics-certs-default" not found Jul 13 17:05:39.236841 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:39.236771 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57b96b57-d3e5-44f3-bcca-2a077820b845-metrics-tls podName:57b96b57-d3e5-44f3-bcca-2a077820b845 nodeName:}" failed. No retries permitted until 2026-07-13 17:05:41.236762031 +0000 UTC m=+37.634420389 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/57b96b57-d3e5-44f3-bcca-2a077820b845-metrics-tls") pod "dns-default-9xwzd" (UID: "57b96b57-d3e5-44f3-bcca-2a077820b845") : secret "dns-default-metrics-tls" not found Jul 13 17:05:39.236841 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:39.236786 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e443fc05-8996-474b-85fa-f3a28463785a-service-ca-bundle podName:e443fc05-8996-474b-85fa-f3a28463785a nodeName:}" failed. No retries permitted until 2026-07-13 17:05:41.236778548 +0000 UTC m=+37.634436907 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e443fc05-8996-474b-85fa-f3a28463785a-service-ca-bundle") pod "router-default-969d568db-zxb2r" (UID: "e443fc05-8996-474b-85fa-f3a28463785a") : configmap references non-existent config key: service-ca.crt Jul 13 17:05:39.267479 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:39.267453 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2b6wx"] Jul 13 17:05:39.270202 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:39.270181 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d1dd68a_166a_4be1_812e_3e15e056c3f2.slice/crio-0c88db0224e6428ee04019668e933bdafa7dffa66d440eb395d68079cd28f51c WatchSource:0}: Error finding container 0c88db0224e6428ee04019668e933bdafa7dffa66d440eb395d68079cd28f51c: Status 404 returned error can't find the container with id 0c88db0224e6428ee04019668e933bdafa7dffa66d440eb395d68079cd28f51c Jul 13 17:05:39.288581 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:39.288541 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5799c95bf4-xw7dg" event={"ID":"5cbb7ab6-e43d-4ef7-b9f2-c2de3288d7ca","Type":"ContainerStarted","Data":"075d17e04e8dedb54c02dd057b9885fabefd736ba549cf0c1352045fce3e816f"} Jul 13 17:05:39.289583 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:39.289548 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-fskzj" event={"ID":"ddbfed7e-ef66-42cf-a6e3-481f1da70624","Type":"ContainerStarted","Data":"ceb9342736f472fc0b2c7aadd8237b4236c130e39d83500429828a78e289b156"} Jul 13 17:05:39.290521 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:39.290503 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b8897fcc8-fl8sl" event={"ID":"de14e3bd-ed7a-4a6c-80c5-0a631a9301d2","Type":"ContainerStarted","Data":"d43ea3b90989b4633f489be297ae7b5460161bdfa3c219b2b99cefb7584480ca"} Jul 13 17:05:39.291537 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:39.291518 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc7d8764f-h64lg" event={"ID":"e3896c87-e2a0-4309-aa5d-bcf65d9e67e9","Type":"ContainerStarted","Data":"c9a1440dae8fc4c791660db06805cf68b2da5a53b813de7eed873795fee7de36"} Jul 13 17:05:39.292542 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:39.292523 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-7s9dh" event={"ID":"bd4ce4fa-6339-4a0c-8260-c6a9538543c2","Type":"ContainerStarted","Data":"d0d4c488ee32d895822c936166a2f24083a378534a2803cf18d055f908fcbbd5"} Jul 13 17:05:39.293500 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:39.293469 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2b6wx" event={"ID":"5d1dd68a-166a-4be1-812e-3e15e056c3f2","Type":"ContainerStarted","Data":"0c88db0224e6428ee04019668e933bdafa7dffa66d440eb395d68079cd28f51c"} Jul 13 17:05:39.294484 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:39.294462 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-rhgb9" event={"ID":"2bfc2ce7-6ebd-4197-86fa-19d9e2b385c8","Type":"ContainerStarted","Data":"82e10cae1901baccbdf2b603800dc325ceac8aa983e60eefc029cf76970ddb69"} Jul 13 17:05:39.295478 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:39.295460 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5s55t" event={"ID":"2f125df3-6cb8-4ec4-bd2f-24428ce6127e","Type":"ContainerStarted","Data":"9f3e5d3fa268bfb3aa0e5ac4d85a794135580b861d15990638b3fe66ec2fe315"} Jul 13 17:05:39.296579 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:39.296544 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-hzcql" event={"ID":"c40e6e88-ab3f-4e3c-a0e7-2400bda333eb","Type":"ContainerStarted","Data":"d5bd0b198d1c4d139b15ba7e0e629b1ed64d0b0b605743dada77dfe2d6945ade"} Jul 13 17:05:39.299357 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:39.299326 2569 generic.go:356] "Generic (PLEG): container finished" podID="921a5cfd-83c7-4db7-be13-53e44c5df655" containerID="fd936b08d45debacecb156086ffd564e62553991298097f601c439e9b326468b" exitCode=0 Jul 13 17:05:39.299450 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:39.299404 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xpws4" event={"ID":"921a5cfd-83c7-4db7-be13-53e44c5df655","Type":"ContainerDied","Data":"fd936b08d45debacecb156086ffd564e62553991298097f601c439e9b326468b"} Jul 13 17:05:39.300530 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:39.300513 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-j2xl7" event={"ID":"7ffdcb20-b7c8-468e-9c1f-c68e9e588a93","Type":"ContainerStarted","Data":"891bf33f8462f58420ce9945561e553699cf0b0a7becfeb2b8710c052b73deaf"} Jul 13 17:05:39.301621 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:39.301604 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-hq4vq" event={"ID":"2eb28be8-8f93-44e3-b09b-466737efa1b2","Type":"ContainerStarted","Data":"caf0f9eb79e27b754b8a6bd3bd8f2c424c94e964a82e2d779a87e7236be1a6c9"} Jul 13 17:05:40.355994 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:40.354882 2569 generic.go:356] "Generic (PLEG): container finished" podID="921a5cfd-83c7-4db7-be13-53e44c5df655" containerID="d8e59de28fb0cbb24e9b15f9ff9f2cb75ea3deb3fc7113289b38a19e0626d9c0" exitCode=0 Jul 13 17:05:40.355994 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:40.354951 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xpws4" event={"ID":"921a5cfd-83c7-4db7-be13-53e44c5df655","Type":"ContainerDied","Data":"d8e59de28fb0cbb24e9b15f9ff9f2cb75ea3deb3fc7113289b38a19e0626d9c0"} Jul 13 17:05:41.158717 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:41.158454 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/46bae189-2ca3-416c-bcb7-42b1283f71b8-registry-tls\") pod \"image-registry-5878cf7589-sgggs\" (UID: \"46bae189-2ca3-416c-bcb7-42b1283f71b8\") " pod="openshift-image-registry/image-registry-5878cf7589-sgggs" Jul 13 17:05:41.158886 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:41.158775 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1a2cb5a4-a8a6-4519-b001-9606305fc3a1-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-lzd48\" (UID: \"1a2cb5a4-a8a6-4519-b001-9606305fc3a1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-lzd48" Jul 13 17:05:41.158886 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:41.158857 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7549ca91-bd0b-4b2f-b76c-0a7784f2fd4c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-ttd4d\" (UID: \"7549ca91-bd0b-4b2f-b76c-0a7784f2fd4c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-ttd4d" Jul 13 17:05:41.159005 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:41.158995 2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 13 17:05:41.159076 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:41.159061 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7549ca91-bd0b-4b2f-b76c-0a7784f2fd4c-cluster-monitoring-operator-tls podName:7549ca91-bd0b-4b2f-b76c-0a7784f2fd4c nodeName:}" failed. No retries permitted until 2026-07-13 17:05:45.159041518 +0000 UTC m=+41.556699875 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7549ca91-bd0b-4b2f-b76c-0a7784f2fd4c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-ttd4d" (UID: "7549ca91-bd0b-4b2f-b76c-0a7784f2fd4c") : secret "cluster-monitoring-operator-tls" not found Jul 13 17:05:41.159833 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:41.159491 2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 13 17:05:41.159833 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:41.159510 2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5878cf7589-sgggs: secret "image-registry-tls" not found Jul 13 17:05:41.159833 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:41.159574 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/46bae189-2ca3-416c-bcb7-42b1283f71b8-registry-tls podName:46bae189-2ca3-416c-bcb7-42b1283f71b8 nodeName:}" failed. No retries permitted until 2026-07-13 17:05:45.1595428 +0000 UTC m=+41.557201161 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/46bae189-2ca3-416c-bcb7-42b1283f71b8-registry-tls") pod "image-registry-5878cf7589-sgggs" (UID: "46bae189-2ca3-416c-bcb7-42b1283f71b8") : secret "image-registry-tls" not found Jul 13 17:05:41.159833 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:41.159636 2569 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 13 17:05:41.159833 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:41.159668 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a2cb5a4-a8a6-4519-b001-9606305fc3a1-samples-operator-tls podName:1a2cb5a4-a8a6-4519-b001-9606305fc3a1 nodeName:}" failed. No retries permitted until 2026-07-13 17:05:45.159657472 +0000 UTC m=+41.557315834 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1a2cb5a4-a8a6-4519-b001-9606305fc3a1-samples-operator-tls") pod "cluster-samples-operator-65db858894-lzd48" (UID: "1a2cb5a4-a8a6-4519-b001-9606305fc3a1") : secret "samples-operator-tls" not found Jul 13 17:05:41.260501 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:41.259459 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/afb76ab4-2a66-45a7-8c98-7c3ca6e026b9-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-5wvjq\" (UID: \"afb76ab4-2a66-45a7-8c98-7c3ca6e026b9\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-5wvjq" Jul 13 17:05:41.260501 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:41.259538 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/57b96b57-d3e5-44f3-bcca-2a077820b845-metrics-tls\") pod \"dns-default-9xwzd\" (UID: \"57b96b57-d3e5-44f3-bcca-2a077820b845\") " pod="openshift-dns/dns-default-9xwzd" Jul 13 17:05:41.260501 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:41.259592 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e443fc05-8996-474b-85fa-f3a28463785a-service-ca-bundle\") pod \"router-default-969d568db-zxb2r\" (UID: \"e443fc05-8996-474b-85fa-f3a28463785a\") " pod="openshift-ingress/router-default-969d568db-zxb2r" Jul 13 17:05:41.260501 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:41.259668 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e443fc05-8996-474b-85fa-f3a28463785a-metrics-certs\") pod \"router-default-969d568db-zxb2r\" (UID: \"e443fc05-8996-474b-85fa-f3a28463785a\") " pod="openshift-ingress/router-default-969d568db-zxb2r" Jul 13 17:05:41.260501 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:41.259804 2569 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 13 17:05:41.260501 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:41.259865 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e443fc05-8996-474b-85fa-f3a28463785a-metrics-certs podName:e443fc05-8996-474b-85fa-f3a28463785a nodeName:}" failed. No retries permitted until 2026-07-13 17:05:45.259845736 +0000 UTC m=+41.657504096 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e443fc05-8996-474b-85fa-f3a28463785a-metrics-certs") pod "router-default-969d568db-zxb2r" (UID: "e443fc05-8996-474b-85fa-f3a28463785a") : secret "router-metrics-certs-default" not found Jul 13 17:05:41.260501 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:41.260241 2569 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 13 17:05:41.260501 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:41.260291 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afb76ab4-2a66-45a7-8c98-7c3ca6e026b9-networking-console-plugin-cert podName:afb76ab4-2a66-45a7-8c98-7c3ca6e026b9 nodeName:}" failed. No retries permitted until 2026-07-13 17:05:45.260276725 +0000 UTC m=+41.657935085 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/afb76ab4-2a66-45a7-8c98-7c3ca6e026b9-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-5wvjq" (UID: "afb76ab4-2a66-45a7-8c98-7c3ca6e026b9") : secret "networking-console-plugin-cert" not found Jul 13 17:05:41.260501 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:41.260346 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 17:05:41.260501 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:41.260376 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57b96b57-d3e5-44f3-bcca-2a077820b845-metrics-tls podName:57b96b57-d3e5-44f3-bcca-2a077820b845 nodeName:}" failed. No retries permitted until 2026-07-13 17:05:45.260366182 +0000 UTC m=+41.658024540 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/57b96b57-d3e5-44f3-bcca-2a077820b845-metrics-tls") pod "dns-default-9xwzd" (UID: "57b96b57-d3e5-44f3-bcca-2a077820b845") : secret "dns-default-metrics-tls" not found Jul 13 17:05:41.260501 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:41.260447 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e443fc05-8996-474b-85fa-f3a28463785a-service-ca-bundle podName:e443fc05-8996-474b-85fa-f3a28463785a nodeName:}" failed. No retries permitted until 2026-07-13 17:05:45.260437373 +0000 UTC m=+41.658095734 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e443fc05-8996-474b-85fa-f3a28463785a-service-ca-bundle") pod "router-default-969d568db-zxb2r" (UID: "e443fc05-8996-474b-85fa-f3a28463785a") : configmap references non-existent config key: service-ca.crt Jul 13 17:05:41.431280 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:41.431185 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xpws4" event={"ID":"921a5cfd-83c7-4db7-be13-53e44c5df655","Type":"ContainerStarted","Data":"26c35e67fdcff8a2f4e15f6dd2c8f91dd26c8b98c99ddfa3f2101a0e13b8692b"} Jul 13 17:05:41.455210 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:41.455155 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-xpws4" podStartSLOduration=8.337292217 podStartE2EDuration="37.455133298s" totalImagesPullingTime="29.117841081s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 17:05:04 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:05:41.453070171 +0000 UTC m=+37.850728552" watchObservedRunningTime="2026-07-13 17:05:41.455133298 +0000 UTC m=+37.852791680" Jul 13 17:05:41.580735 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:41.580558 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-phcxr_b3ec2a99-eadb-4865-ae2a-dd77aab461de/konnectivity-agent/0.log" limit=1024 Jul 13 17:05:41.641940 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:41.641821 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-phcxr_b3ec2a99-eadb-4865-ae2a-dd77aab461de/konnectivity-agent/0.log" limit=1024 Jul 13 17:05:43.572859 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:43.572745 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-phcxr_b3ec2a99-eadb-4865-ae2a-dd77aab461de/konnectivity-agent/0.log" limit=1024 Jul 13 17:05:43.628014 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:43.627898 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-phcxr_b3ec2a99-eadb-4865-ae2a-dd77aab461de/konnectivity-agent/0.log" limit=1024 Jul 13 17:05:45.149132 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:45.149090 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-phcxr_b3ec2a99-eadb-4865-ae2a-dd77aab461de/konnectivity-agent/0.log" limit=1024 Jul 13 17:05:45.203183 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:45.203144 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7549ca91-bd0b-4b2f-b76c-0a7784f2fd4c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-ttd4d\" (UID: \"7549ca91-bd0b-4b2f-b76c-0a7784f2fd4c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-ttd4d" Jul 13 17:05:45.203338 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:45.203279 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/46bae189-2ca3-416c-bcb7-42b1283f71b8-registry-tls\") pod \"image-registry-5878cf7589-sgggs\" (UID: \"46bae189-2ca3-416c-bcb7-42b1283f71b8\") " pod="openshift-image-registry/image-registry-5878cf7589-sgggs" Jul 13 17:05:45.203338 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:45.203322 2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 13 17:05:45.203457 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:45.203332 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1a2cb5a4-a8a6-4519-b001-9606305fc3a1-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-lzd48\" (UID: \"1a2cb5a4-a8a6-4519-b001-9606305fc3a1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-lzd48" Jul 13 17:05:45.203457 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:45.203412 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7549ca91-bd0b-4b2f-b76c-0a7784f2fd4c-cluster-monitoring-operator-tls podName:7549ca91-bd0b-4b2f-b76c-0a7784f2fd4c nodeName:}" failed. No retries permitted until 2026-07-13 17:05:53.203389061 +0000 UTC m=+49.601047419 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7549ca91-bd0b-4b2f-b76c-0a7784f2fd4c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-ttd4d" (UID: "7549ca91-bd0b-4b2f-b76c-0a7784f2fd4c") : secret "cluster-monitoring-operator-tls" not found Jul 13 17:05:45.203457 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:45.203416 2569 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 13 17:05:45.203457 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:45.203423 2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 13 17:05:45.203457 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:45.203441 2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5878cf7589-sgggs: secret "image-registry-tls" not found Jul 13 17:05:45.203668 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:45.203468 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a2cb5a4-a8a6-4519-b001-9606305fc3a1-samples-operator-tls podName:1a2cb5a4-a8a6-4519-b001-9606305fc3a1 nodeName:}" failed. No retries permitted until 2026-07-13 17:05:53.203456468 +0000 UTC m=+49.601114825 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1a2cb5a4-a8a6-4519-b001-9606305fc3a1-samples-operator-tls") pod "cluster-samples-operator-65db858894-lzd48" (UID: "1a2cb5a4-a8a6-4519-b001-9606305fc3a1") : secret "samples-operator-tls" not found Jul 13 17:05:45.203668 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:45.203499 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/46bae189-2ca3-416c-bcb7-42b1283f71b8-registry-tls podName:46bae189-2ca3-416c-bcb7-42b1283f71b8 nodeName:}" failed. No retries permitted until 2026-07-13 17:05:53.203481798 +0000 UTC m=+49.601140161 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/46bae189-2ca3-416c-bcb7-42b1283f71b8-registry-tls") pod "image-registry-5878cf7589-sgggs" (UID: "46bae189-2ca3-416c-bcb7-42b1283f71b8") : secret "image-registry-tls" not found Jul 13 17:05:45.304895 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:45.304859 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/57b96b57-d3e5-44f3-bcca-2a077820b845-metrics-tls\") pod \"dns-default-9xwzd\" (UID: \"57b96b57-d3e5-44f3-bcca-2a077820b845\") " pod="openshift-dns/dns-default-9xwzd" Jul 13 17:05:45.305050 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:45.304923 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e443fc05-8996-474b-85fa-f3a28463785a-service-ca-bundle\") pod \"router-default-969d568db-zxb2r\" (UID: \"e443fc05-8996-474b-85fa-f3a28463785a\") " pod="openshift-ingress/router-default-969d568db-zxb2r" Jul 13 17:05:45.305050 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:45.305001 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e443fc05-8996-474b-85fa-f3a28463785a-metrics-certs\") pod \"router-default-969d568db-zxb2r\" (UID: \"e443fc05-8996-474b-85fa-f3a28463785a\") " pod="openshift-ingress/router-default-969d568db-zxb2r" Jul 13 17:05:45.305146 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:45.305079 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 17:05:45.305146 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:45.305094 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e443fc05-8996-474b-85fa-f3a28463785a-service-ca-bundle podName:e443fc05-8996-474b-85fa-f3a28463785a nodeName:}" failed. No retries permitted until 2026-07-13 17:05:53.305071213 +0000 UTC m=+49.702729593 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e443fc05-8996-474b-85fa-f3a28463785a-service-ca-bundle") pod "router-default-969d568db-zxb2r" (UID: "e443fc05-8996-474b-85fa-f3a28463785a") : configmap references non-existent config key: service-ca.crt Jul 13 17:05:45.305146 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:45.305135 2569 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 13 17:05:45.305291 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:45.305130 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/afb76ab4-2a66-45a7-8c98-7c3ca6e026b9-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-5wvjq\" (UID: \"afb76ab4-2a66-45a7-8c98-7c3ca6e026b9\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-5wvjq" Jul 13 17:05:45.305291 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:45.305163 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57b96b57-d3e5-44f3-bcca-2a077820b845-metrics-tls podName:57b96b57-d3e5-44f3-bcca-2a077820b845 nodeName:}" failed. No retries permitted until 2026-07-13 17:05:53.305146473 +0000 UTC m=+49.702804832 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/57b96b57-d3e5-44f3-bcca-2a077820b845-metrics-tls") pod "dns-default-9xwzd" (UID: "57b96b57-d3e5-44f3-bcca-2a077820b845") : secret "dns-default-metrics-tls" not found Jul 13 17:05:45.305291 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:45.305191 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e443fc05-8996-474b-85fa-f3a28463785a-metrics-certs podName:e443fc05-8996-474b-85fa-f3a28463785a nodeName:}" failed. No retries permitted until 2026-07-13 17:05:53.305177246 +0000 UTC m=+49.702835607 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e443fc05-8996-474b-85fa-f3a28463785a-metrics-certs") pod "router-default-969d568db-zxb2r" (UID: "e443fc05-8996-474b-85fa-f3a28463785a") : secret "router-metrics-certs-default" not found Jul 13 17:05:45.305291 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:45.305198 2569 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 13 17:05:45.305291 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:45.305241 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afb76ab4-2a66-45a7-8c98-7c3ca6e026b9-networking-console-plugin-cert podName:afb76ab4-2a66-45a7-8c98-7c3ca6e026b9 nodeName:}" failed. No retries permitted until 2026-07-13 17:05:53.305228825 +0000 UTC m=+49.702887185 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/afb76ab4-2a66-45a7-8c98-7c3ca6e026b9-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-5wvjq" (UID: "afb76ab4-2a66-45a7-8c98-7c3ca6e026b9") : secret "networking-console-plugin-cert" not found Jul 13 17:05:53.280909 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:53.280595 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7549ca91-bd0b-4b2f-b76c-0a7784f2fd4c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-ttd4d\" (UID: \"7549ca91-bd0b-4b2f-b76c-0a7784f2fd4c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-ttd4d" Jul 13 17:05:53.280909 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:53.280693 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/46bae189-2ca3-416c-bcb7-42b1283f71b8-registry-tls\") pod \"image-registry-5878cf7589-sgggs\" (UID: \"46bae189-2ca3-416c-bcb7-42b1283f71b8\") " pod="openshift-image-registry/image-registry-5878cf7589-sgggs" Jul 13 17:05:53.280909 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:53.280748 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1a2cb5a4-a8a6-4519-b001-9606305fc3a1-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-lzd48\" (UID: \"1a2cb5a4-a8a6-4519-b001-9606305fc3a1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-lzd48" Jul 13 17:05:53.280909 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:53.280895 2569 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 13 17:05:53.281934 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:53.280949 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a2cb5a4-a8a6-4519-b001-9606305fc3a1-samples-operator-tls podName:1a2cb5a4-a8a6-4519-b001-9606305fc3a1 nodeName:}" failed. No retries permitted until 2026-07-13 17:06:09.280930908 +0000 UTC m=+65.678589284 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1a2cb5a4-a8a6-4519-b001-9606305fc3a1-samples-operator-tls") pod "cluster-samples-operator-65db858894-lzd48" (UID: "1a2cb5a4-a8a6-4519-b001-9606305fc3a1") : secret "samples-operator-tls" not found Jul 13 17:05:53.281934 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:53.281044 2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 13 17:05:53.281934 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:53.281061 2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5878cf7589-sgggs: secret "image-registry-tls" not found Jul 13 17:05:53.281934 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:53.281089 2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 13 17:05:53.281934 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:53.281103 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/46bae189-2ca3-416c-bcb7-42b1283f71b8-registry-tls podName:46bae189-2ca3-416c-bcb7-42b1283f71b8 nodeName:}" failed. No retries permitted until 2026-07-13 17:06:09.281086498 +0000 UTC m=+65.678744857 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/46bae189-2ca3-416c-bcb7-42b1283f71b8-registry-tls") pod "image-registry-5878cf7589-sgggs" (UID: "46bae189-2ca3-416c-bcb7-42b1283f71b8") : secret "image-registry-tls" not found Jul 13 17:05:53.281934 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:53.281132 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7549ca91-bd0b-4b2f-b76c-0a7784f2fd4c-cluster-monitoring-operator-tls podName:7549ca91-bd0b-4b2f-b76c-0a7784f2fd4c nodeName:}" failed. No retries permitted until 2026-07-13 17:06:09.281118747 +0000 UTC m=+65.678777124 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7549ca91-bd0b-4b2f-b76c-0a7784f2fd4c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-ttd4d" (UID: "7549ca91-bd0b-4b2f-b76c-0a7784f2fd4c") : secret "cluster-monitoring-operator-tls" not found Jul 13 17:05:53.382299 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:53.382198 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/57b96b57-d3e5-44f3-bcca-2a077820b845-metrics-tls\") pod \"dns-default-9xwzd\" (UID: \"57b96b57-d3e5-44f3-bcca-2a077820b845\") " pod="openshift-dns/dns-default-9xwzd" Jul 13 17:05:53.382299 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:53.382250 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e443fc05-8996-474b-85fa-f3a28463785a-service-ca-bundle\") pod \"router-default-969d568db-zxb2r\" (UID: \"e443fc05-8996-474b-85fa-f3a28463785a\") " pod="openshift-ingress/router-default-969d568db-zxb2r" Jul 13 17:05:53.382479 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:53.382313 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e443fc05-8996-474b-85fa-f3a28463785a-metrics-certs\") pod \"router-default-969d568db-zxb2r\" (UID: \"e443fc05-8996-474b-85fa-f3a28463785a\") " pod="openshift-ingress/router-default-969d568db-zxb2r" Jul 13 17:05:53.382479 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:53.382390 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/afb76ab4-2a66-45a7-8c98-7c3ca6e026b9-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-5wvjq\" (UID: \"afb76ab4-2a66-45a7-8c98-7c3ca6e026b9\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-5wvjq" Jul 13 17:05:53.382595 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:53.382537 2569 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 13 17:05:53.382654 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:53.382613 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afb76ab4-2a66-45a7-8c98-7c3ca6e026b9-networking-console-plugin-cert podName:afb76ab4-2a66-45a7-8c98-7c3ca6e026b9 nodeName:}" failed. No retries permitted until 2026-07-13 17:06:09.38259297 +0000 UTC m=+65.780251331 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/afb76ab4-2a66-45a7-8c98-7c3ca6e026b9-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-5wvjq" (UID: "afb76ab4-2a66-45a7-8c98-7c3ca6e026b9") : secret "networking-console-plugin-cert" not found Jul 13 17:05:53.382788 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:53.382677 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 17:05:53.382788 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:53.382705 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57b96b57-d3e5-44f3-bcca-2a077820b845-metrics-tls podName:57b96b57-d3e5-44f3-bcca-2a077820b845 nodeName:}" failed. No retries permitted until 2026-07-13 17:06:09.382695094 +0000 UTC m=+65.780353453 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/57b96b57-d3e5-44f3-bcca-2a077820b845-metrics-tls") pod "dns-default-9xwzd" (UID: "57b96b57-d3e5-44f3-bcca-2a077820b845") : secret "dns-default-metrics-tls" not found Jul 13 17:05:53.382788 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:53.382770 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e443fc05-8996-474b-85fa-f3a28463785a-service-ca-bundle podName:e443fc05-8996-474b-85fa-f3a28463785a nodeName:}" failed. No retries permitted until 2026-07-13 17:06:09.382760928 +0000 UTC m=+65.780419289 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e443fc05-8996-474b-85fa-f3a28463785a-service-ca-bundle") pod "router-default-969d568db-zxb2r" (UID: "e443fc05-8996-474b-85fa-f3a28463785a") : configmap references non-existent config key: service-ca.crt Jul 13 17:05:53.382996 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:53.382823 2569 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 13 17:05:53.382996 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:53.382848 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e443fc05-8996-474b-85fa-f3a28463785a-metrics-certs podName:e443fc05-8996-474b-85fa-f3a28463785a nodeName:}" failed. No retries permitted until 2026-07-13 17:06:09.382839665 +0000 UTC m=+65.780498025 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e443fc05-8996-474b-85fa-f3a28463785a-metrics-certs") pod "router-default-969d568db-zxb2r" (UID: "e443fc05-8996-474b-85fa-f3a28463785a") : secret "router-metrics-certs-default" not found Jul 13 17:05:53.486257 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:53.486228 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2b6wx" event={"ID":"5d1dd68a-166a-4be1-812e-3e15e056c3f2","Type":"ContainerStarted","Data":"7c6295db3e060a2f870682b69e8e3cab576af8cec25f18a42530b4dd244d839c"} Jul 13 17:05:53.486381 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:53.486319 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-2b6wx" Jul 13 17:05:53.488363 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:53.488337 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-rhgb9" event={"ID":"2bfc2ce7-6ebd-4197-86fa-19d9e2b385c8","Type":"ContainerStarted","Data":"61ff96cab18304627d1f4c6f74d6c98a49bf8344476317c1f321f16ef3e99ad0"} Jul 13 17:05:53.491159 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:53.491122 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5s55t" event={"ID":"2f125df3-6cb8-4ec4-bd2f-24428ce6127e","Type":"ContainerStarted","Data":"47980a7d9da279de3493ac1fcf621ce5e1732c5a220e1cbd4fbc4d809d853e1e"} Jul 13 17:05:53.493105 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:53.493086 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-hzcql_c40e6e88-ab3f-4e3c-a0e7-2400bda333eb/console-operator/0.log" Jul 13 17:05:53.493192 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:53.493125 2569 generic.go:356] "Generic (PLEG): container finished" podID="c40e6e88-ab3f-4e3c-a0e7-2400bda333eb" containerID="8bc6a141e26fa69133d25d4a53e3f002977e0902890d5d7d5fa926ed368966f1" exitCode=255 Jul 13 17:05:53.493250 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:53.493204 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-hzcql" event={"ID":"c40e6e88-ab3f-4e3c-a0e7-2400bda333eb","Type":"ContainerDied","Data":"8bc6a141e26fa69133d25d4a53e3f002977e0902890d5d7d5fa926ed368966f1"} Jul 13 17:05:53.493495 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:53.493461 2569 scope.go:117] "RemoveContainer" containerID="8bc6a141e26fa69133d25d4a53e3f002977e0902890d5d7d5fa926ed368966f1" Jul 13 17:05:53.496986 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:53.496902 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-j2xl7" event={"ID":"7ffdcb20-b7c8-468e-9c1f-c68e9e588a93","Type":"ContainerStarted","Data":"23c05a194122135fa1770163e0b03ae06e86935515b0e7781c1519b017bb0788"} Jul 13 17:05:53.500133 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:53.500111 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-hq4vq" event={"ID":"2eb28be8-8f93-44e3-b09b-466737efa1b2","Type":"ContainerStarted","Data":"192b5046124b60beecd94358521acb624a527edcc7ae27852b8ca0c8900992d5"} Jul 13 17:05:53.502244 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:53.502197 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-2b6wx" podStartSLOduration=35.929297324 podStartE2EDuration="49.502181073s" totalImagesPullingTime="13.572883749s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:05:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:05:53.501597366 +0000 UTC m=+49.899255752" watchObservedRunningTime="2026-07-13 17:05:53.502181073 +0000 UTC m=+49.899839453" Jul 13 17:05:53.511136 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:53.507736 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5799c95bf4-xw7dg" event={"ID":"5cbb7ab6-e43d-4ef7-b9f2-c2de3288d7ca","Type":"ContainerStarted","Data":"a36dd6550bf0c2cee9c57d23e3fe3430f83f4c796f8cf0a256b00bc05f3be0a1"} Jul 13 17:05:53.511136 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:53.509131 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5799c95bf4-xw7dg" Jul 13 17:05:53.511301 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:53.511269 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5799c95bf4-xw7dg" Jul 13 17:05:53.520332 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:53.520286 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-rhgb9" podStartSLOduration=20.907730815 podStartE2EDuration="29.520267936s" totalImagesPullingTime="8.612537121s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:05:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:05:53.519150241 +0000 UTC m=+49.916808620" watchObservedRunningTime="2026-07-13 17:05:53.520267936 +0000 UTC m=+49.917926316" Jul 13 17:05:53.522657 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:53.522634 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-fskzj" event={"ID":"ddbfed7e-ef66-42cf-a6e3-481f1da70624","Type":"ContainerStarted","Data":"21b53a10c667b1a9efb99a3b032ebee4c6837a9ff3101be17143f6458bd4712e"} Jul 13 17:05:53.525299 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:53.525274 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b8897fcc8-fl8sl" event={"ID":"de14e3bd-ed7a-4a6c-80c5-0a631a9301d2","Type":"ContainerStarted","Data":"7446eb6590938460b72a1c493d7b63be723b0728613aefe3623e882bb90ad38c"} Jul 13 17:05:53.528114 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:53.528091 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc7d8764f-h64lg" event={"ID":"e3896c87-e2a0-4309-aa5d-bcf65d9e67e9","Type":"ContainerStarted","Data":"78f9b91c2781ba9bc6520f9d1b74851e4f350057c2d456352410230cd7d046e6"} Jul 13 17:05:53.528209 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:53.528129 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc7d8764f-h64lg" event={"ID":"e3896c87-e2a0-4309-aa5d-bcf65d9e67e9","Type":"ContainerStarted","Data":"f58ffc9acf589970e13a69d40e17dcc8b01a4f7634befcb9ac831f5fb5590e39"} Jul 13 17:05:53.534591 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:53.534537 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-7s9dh" event={"ID":"bd4ce4fa-6339-4a0c-8260-c6a9538543c2","Type":"ContainerStarted","Data":"fe15154c254dfa806e3a2628d929864811cf5705e9f217c03e41a00ea808444c"} Jul 13 17:05:53.536691 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:53.536651 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-j2xl7" podStartSLOduration=15.685435062 podStartE2EDuration="29.536639934s" totalImagesPullingTime="13.851204872s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:05:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:05:53.536280437 +0000 UTC m=+49.933938812" watchObservedRunningTime="2026-07-13 17:05:53.536639934 +0000 UTC m=+49.934298313" Jul 13 17:05:53.605919 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:53.605812 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-hq4vq" podStartSLOduration=15.776571903 podStartE2EDuration="29.605767998s" totalImagesPullingTime="13.829196095s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:05:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:05:53.605285019 +0000 UTC m=+50.002943399" watchObservedRunningTime="2026-07-13 17:05:53.605767998 +0000 UTC m=+50.003426379" Jul 13 17:05:53.607078 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:53.606734 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-5s55t" podStartSLOduration=2.817931611 podStartE2EDuration="16.606723823s" totalImagesPullingTime="13.788792212s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:05:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:05:53.576163093 +0000 UTC m=+49.973821467" watchObservedRunningTime="2026-07-13 17:05:53.606723823 +0000 UTC m=+50.004382183" Jul 13 17:05:53.625177 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:53.624246 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-7s9dh" podStartSLOduration=15.781281658 podStartE2EDuration="29.624227458s" totalImagesPullingTime="13.8429458s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:05:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:05:53.622353631 +0000 UTC m=+50.020012036" watchObservedRunningTime="2026-07-13 17:05:53.624227458 +0000 UTC m=+50.021885839" Jul 13 17:05:53.643977 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:53.643933 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-fskzj" podStartSLOduration=17.06987081 podStartE2EDuration="29.643914692s" totalImagesPullingTime="12.574043882s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:05:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:05:53.642812078 +0000 UTC m=+50.040470459" watchObservedRunningTime="2026-07-13 17:05:53.643914692 +0000 UTC m=+50.041573073" Jul 13 17:05:53.660874 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:53.660823 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b8897fcc8-fl8sl" podStartSLOduration=5.793496845 podStartE2EDuration="19.660795982s" totalImagesPullingTime="13.867299137s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:05:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:05:53.659990775 +0000 UTC m=+50.057649180" watchObservedRunningTime="2026-07-13 17:05:53.660795982 +0000 UTC m=+50.058454363" Jul 13 17:05:53.683374 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:53.682749 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5799c95bf4-xw7dg" podStartSLOduration=5.814554116 podStartE2EDuration="19.682731622s" totalImagesPullingTime="13.868177506s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:05:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:05:53.681710058 +0000 UTC m=+50.079368451" watchObservedRunningTime="2026-07-13 17:05:53.682731622 +0000 UTC m=+50.080390003" Jul 13 17:05:54.289091 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:54.289053 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-cfcv4"] Jul 13 17:05:54.309709 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:54.309678 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-cfcv4"] Jul 13 17:05:54.309883 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:54.309818 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cfcv4" Jul 13 17:05:54.312244 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:54.312219 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 13 17:05:54.312400 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:54.312384 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 13 17:05:54.312526 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:54.312506 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rmd8z\"" Jul 13 17:05:54.394556 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:54.394525 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/965ad197-c001-48c3-9ffd-58b4d7415d1e-crio-socket\") pod \"insights-runtime-extractor-cfcv4\" (UID: \"965ad197-c001-48c3-9ffd-58b4d7415d1e\") " pod="openshift-insights/insights-runtime-extractor-cfcv4" Jul 13 17:05:54.394556 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:54.394556 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/965ad197-c001-48c3-9ffd-58b4d7415d1e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cfcv4\" (UID: \"965ad197-c001-48c3-9ffd-58b4d7415d1e\") " pod="openshift-insights/insights-runtime-extractor-cfcv4" Jul 13 17:05:54.394818 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:54.394597 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/965ad197-c001-48c3-9ffd-58b4d7415d1e-data-volume\") pod \"insights-runtime-extractor-cfcv4\" (UID: \"965ad197-c001-48c3-9ffd-58b4d7415d1e\") " pod="openshift-insights/insights-runtime-extractor-cfcv4" Jul 13 17:05:54.394818 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:54.394715 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4crq9\" (UniqueName: \"kubernetes.io/projected/965ad197-c001-48c3-9ffd-58b4d7415d1e-kube-api-access-4crq9\") pod \"insights-runtime-extractor-cfcv4\" (UID: \"965ad197-c001-48c3-9ffd-58b4d7415d1e\") " pod="openshift-insights/insights-runtime-extractor-cfcv4" Jul 13 17:05:54.394933 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:54.394881 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/965ad197-c001-48c3-9ffd-58b4d7415d1e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cfcv4\" (UID: \"965ad197-c001-48c3-9ffd-58b4d7415d1e\") " pod="openshift-insights/insights-runtime-extractor-cfcv4" Jul 13 17:05:54.495339 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:54.495303 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/965ad197-c001-48c3-9ffd-58b4d7415d1e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cfcv4\" (UID: \"965ad197-c001-48c3-9ffd-58b4d7415d1e\") " pod="openshift-insights/insights-runtime-extractor-cfcv4" Jul 13 17:05:54.495509 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:54.495396 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/965ad197-c001-48c3-9ffd-58b4d7415d1e-crio-socket\") pod \"insights-runtime-extractor-cfcv4\" (UID: \"965ad197-c001-48c3-9ffd-58b4d7415d1e\") " pod="openshift-insights/insights-runtime-extractor-cfcv4" Jul 13 17:05:54.495509 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:54.495424 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/965ad197-c001-48c3-9ffd-58b4d7415d1e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cfcv4\" (UID: \"965ad197-c001-48c3-9ffd-58b4d7415d1e\") " pod="openshift-insights/insights-runtime-extractor-cfcv4" Jul 13 17:05:54.495509 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:54.495451 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/965ad197-c001-48c3-9ffd-58b4d7415d1e-data-volume\") pod \"insights-runtime-extractor-cfcv4\" (UID: \"965ad197-c001-48c3-9ffd-58b4d7415d1e\") " pod="openshift-insights/insights-runtime-extractor-cfcv4" Jul 13 17:05:54.495509 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:54.495467 2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 17:05:54.495509 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:54.495500 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4crq9\" (UniqueName: \"kubernetes.io/projected/965ad197-c001-48c3-9ffd-58b4d7415d1e-kube-api-access-4crq9\") pod \"insights-runtime-extractor-cfcv4\" (UID: \"965ad197-c001-48c3-9ffd-58b4d7415d1e\") " pod="openshift-insights/insights-runtime-extractor-cfcv4" Jul 13 17:05:54.495783 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:54.495526 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/965ad197-c001-48c3-9ffd-58b4d7415d1e-insights-runtime-extractor-tls podName:965ad197-c001-48c3-9ffd-58b4d7415d1e nodeName:}" failed. No retries permitted until 2026-07-13 17:05:54.995511193 +0000 UTC m=+51.393169549 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/965ad197-c001-48c3-9ffd-58b4d7415d1e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cfcv4" (UID: "965ad197-c001-48c3-9ffd-58b4d7415d1e") : secret "insights-runtime-extractor-tls" not found Jul 13 17:05:54.495783 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:54.495551 2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 13 17:05:54.495783 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:54.495607 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/965ad197-c001-48c3-9ffd-58b4d7415d1e-crio-socket\") pod \"insights-runtime-extractor-cfcv4\" (UID: \"965ad197-c001-48c3-9ffd-58b4d7415d1e\") " pod="openshift-insights/insights-runtime-extractor-cfcv4" Jul 13 17:05:54.495783 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:54.495625 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/965ad197-c001-48c3-9ffd-58b4d7415d1e-kube-rbac-proxy-cm podName:965ad197-c001-48c3-9ffd-58b4d7415d1e nodeName:}" failed. No retries permitted until 2026-07-13 17:05:54.995607998 +0000 UTC m=+51.393266373 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/965ad197-c001-48c3-9ffd-58b4d7415d1e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cfcv4" (UID: "965ad197-c001-48c3-9ffd-58b4d7415d1e") : configmap "kube-rbac-proxy" not found Jul 13 17:05:54.495996 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:54.495948 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/965ad197-c001-48c3-9ffd-58b4d7415d1e-data-volume\") pod \"insights-runtime-extractor-cfcv4\" (UID: \"965ad197-c001-48c3-9ffd-58b4d7415d1e\") " pod="openshift-insights/insights-runtime-extractor-cfcv4" Jul 13 17:05:54.504987 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:54.504956 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4crq9\" (UniqueName: \"kubernetes.io/projected/965ad197-c001-48c3-9ffd-58b4d7415d1e-kube-api-access-4crq9\") pod \"insights-runtime-extractor-cfcv4\" (UID: \"965ad197-c001-48c3-9ffd-58b4d7415d1e\") " pod="openshift-insights/insights-runtime-extractor-cfcv4" Jul 13 17:05:54.539957 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:54.539905 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-hzcql_c40e6e88-ab3f-4e3c-a0e7-2400bda333eb/console-operator/1.log" Jul 13 17:05:54.540341 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:54.540325 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-hzcql_c40e6e88-ab3f-4e3c-a0e7-2400bda333eb/console-operator/0.log" Jul 13 17:05:54.540405 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:54.540367 2569 generic.go:356] "Generic (PLEG): container finished" podID="c40e6e88-ab3f-4e3c-a0e7-2400bda333eb" containerID="00e88729dcc709480a17df78650156a21d942845bb31323ac69c79ca9b0fe642" exitCode=255 Jul 13 17:05:54.540472 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:54.540450 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-hzcql" event={"ID":"c40e6e88-ab3f-4e3c-a0e7-2400bda333eb","Type":"ContainerDied","Data":"00e88729dcc709480a17df78650156a21d942845bb31323ac69c79ca9b0fe642"} Jul 13 17:05:54.540580 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:54.540550 2569 scope.go:117] "RemoveContainer" containerID="8bc6a141e26fa69133d25d4a53e3f002977e0902890d5d7d5fa926ed368966f1" Jul 13 17:05:54.540772 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:54.540757 2569 scope.go:117] "RemoveContainer" containerID="00e88729dcc709480a17df78650156a21d942845bb31323ac69c79ca9b0fe642" Jul 13 17:05:54.540961 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:54.540938 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-hzcql_openshift-console-operator(c40e6e88-ab3f-4e3c-a0e7-2400bda333eb)\"" pod="openshift-console-operator/console-operator-7866497494-hzcql" podUID="c40e6e88-ab3f-4e3c-a0e7-2400bda333eb" Jul 13 17:05:54.542841 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:54.542789 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc7d8764f-h64lg" event={"ID":"e3896c87-e2a0-4309-aa5d-bcf65d9e67e9","Type":"ContainerStarted","Data":"e302c507a097aaf597af06467f894f4946d6cdda4a218aae9af2e47bb8548299"} Jul 13 17:05:54.574304 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:54.574252 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc7d8764f-h64lg" podStartSLOduration=6.379464113 podStartE2EDuration="20.574239313s" totalImagesPullingTime="14.1947752s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:05:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:05:54.573173025 +0000 UTC m=+50.970831457" watchObservedRunningTime="2026-07-13 17:05:54.574239313 +0000 UTC m=+50.971897692" Jul 13 17:05:54.773648 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:54.773606 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-9pnpw"] Jul 13 17:05:54.782824 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:54.782798 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-9pnpw" Jul 13 17:05:54.787782 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:54.787755 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 13 17:05:54.787782 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:54.787773 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-xl6ng\"" Jul 13 17:05:54.787955 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:54.787893 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 13 17:05:54.788281 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:54.788262 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-9pnpw"] Jul 13 17:05:54.900977 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:54.900948 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lj68c\" (UniqueName: \"kubernetes.io/projected/4e92fb26-ea9b-4a99-bc67-5e52b023aa20-kube-api-access-lj68c\") pod \"migrator-7f59b54f6c-9pnpw\" (UID: \"4e92fb26-ea9b-4a99-bc67-5e52b023aa20\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-9pnpw" Jul 13 17:05:55.001929 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:55.001902 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lj68c\" (UniqueName: \"kubernetes.io/projected/4e92fb26-ea9b-4a99-bc67-5e52b023aa20-kube-api-access-lj68c\") pod \"migrator-7f59b54f6c-9pnpw\" (UID: \"4e92fb26-ea9b-4a99-bc67-5e52b023aa20\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-9pnpw" Jul 13 17:05:55.002048 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:55.001977 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/965ad197-c001-48c3-9ffd-58b4d7415d1e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cfcv4\" (UID: \"965ad197-c001-48c3-9ffd-58b4d7415d1e\") " pod="openshift-insights/insights-runtime-extractor-cfcv4" Jul 13 17:05:55.002048 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:55.002037 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/965ad197-c001-48c3-9ffd-58b4d7415d1e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cfcv4\" (UID: \"965ad197-c001-48c3-9ffd-58b4d7415d1e\") " pod="openshift-insights/insights-runtime-extractor-cfcv4" Jul 13 17:05:55.002119 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:55.002108 2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 13 17:05:55.002150 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:55.002111 2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 17:05:55.002190 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:55.002164 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/965ad197-c001-48c3-9ffd-58b4d7415d1e-kube-rbac-proxy-cm podName:965ad197-c001-48c3-9ffd-58b4d7415d1e nodeName:}" failed. No retries permitted until 2026-07-13 17:05:56.002150542 +0000 UTC m=+52.399808899 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/965ad197-c001-48c3-9ffd-58b4d7415d1e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cfcv4" (UID: "965ad197-c001-48c3-9ffd-58b4d7415d1e") : configmap "kube-rbac-proxy" not found Jul 13 17:05:55.002190 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:55.002179 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/965ad197-c001-48c3-9ffd-58b4d7415d1e-insights-runtime-extractor-tls podName:965ad197-c001-48c3-9ffd-58b4d7415d1e nodeName:}" failed. No retries permitted until 2026-07-13 17:05:56.00217275 +0000 UTC m=+52.399831107 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/965ad197-c001-48c3-9ffd-58b4d7415d1e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cfcv4" (UID: "965ad197-c001-48c3-9ffd-58b4d7415d1e") : secret "insights-runtime-extractor-tls" not found Jul 13 17:05:55.012861 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:55.012830 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lj68c\" (UniqueName: \"kubernetes.io/projected/4e92fb26-ea9b-4a99-bc67-5e52b023aa20-kube-api-access-lj68c\") pod \"migrator-7f59b54f6c-9pnpw\" (UID: \"4e92fb26-ea9b-4a99-bc67-5e52b023aa20\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-9pnpw" Jul 13 17:05:55.093396 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:55.093341 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-9pnpw" Jul 13 17:05:55.225054 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:55.225033 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-9pnpw"] Jul 13 17:05:55.227327 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:55.227302 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e92fb26_ea9b_4a99_bc67_5e52b023aa20.slice/crio-0020c53fd262da982bd020f7bbabda2a9367c14aceb4fc3eb526a39045109c80 WatchSource:0}: Error finding container 0020c53fd262da982bd020f7bbabda2a9367c14aceb4fc3eb526a39045109c80: Status 404 returned error can't find the container with id 0020c53fd262da982bd020f7bbabda2a9367c14aceb4fc3eb526a39045109c80 Jul 13 17:05:55.499094 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:55.499023 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-qn7kc_a7a4b481-50e3-4bb3-bcb5-6c5942814726/dns-node-resolver/0.log" Jul 13 17:05:55.548259 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:55.548223 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-9pnpw" event={"ID":"4e92fb26-ea9b-4a99-bc67-5e52b023aa20","Type":"ContainerStarted","Data":"0020c53fd262da982bd020f7bbabda2a9367c14aceb4fc3eb526a39045109c80"} Jul 13 17:05:55.549780 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:55.549757 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-hzcql_c40e6e88-ab3f-4e3c-a0e7-2400bda333eb/console-operator/1.log" Jul 13 17:05:55.550086 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:55.550067 2569 scope.go:117] "RemoveContainer" containerID="00e88729dcc709480a17df78650156a21d942845bb31323ac69c79ca9b0fe642" Jul 13 17:05:55.550223 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:55.550205 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-hzcql_openshift-console-operator(c40e6e88-ab3f-4e3c-a0e7-2400bda333eb)\"" pod="openshift-console-operator/console-operator-7866497494-hzcql" podUID="c40e6e88-ab3f-4e3c-a0e7-2400bda333eb" Jul 13 17:05:56.012437 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:56.012403 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/965ad197-c001-48c3-9ffd-58b4d7415d1e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cfcv4\" (UID: \"965ad197-c001-48c3-9ffd-58b4d7415d1e\") " pod="openshift-insights/insights-runtime-extractor-cfcv4" Jul 13 17:05:56.012607 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:56.012523 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/965ad197-c001-48c3-9ffd-58b4d7415d1e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cfcv4\" (UID: \"965ad197-c001-48c3-9ffd-58b4d7415d1e\") " pod="openshift-insights/insights-runtime-extractor-cfcv4" Jul 13 17:05:56.012607 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:56.012545 2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 17:05:56.012679 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:56.012625 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/965ad197-c001-48c3-9ffd-58b4d7415d1e-insights-runtime-extractor-tls podName:965ad197-c001-48c3-9ffd-58b4d7415d1e nodeName:}" failed. No retries permitted until 2026-07-13 17:05:58.012607194 +0000 UTC m=+54.410265569 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/965ad197-c001-48c3-9ffd-58b4d7415d1e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cfcv4" (UID: "965ad197-c001-48c3-9ffd-58b4d7415d1e") : secret "insights-runtime-extractor-tls" not found Jul 13 17:05:56.012679 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:56.012664 2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 13 17:05:56.012749 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:56.012711 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/965ad197-c001-48c3-9ffd-58b4d7415d1e-kube-rbac-proxy-cm podName:965ad197-c001-48c3-9ffd-58b4d7415d1e nodeName:}" failed. No retries permitted until 2026-07-13 17:05:58.012698861 +0000 UTC m=+54.410357233 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/965ad197-c001-48c3-9ffd-58b4d7415d1e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cfcv4" (UID: "965ad197-c001-48c3-9ffd-58b4d7415d1e") : configmap "kube-rbac-proxy" not found Jul 13 17:05:56.299452 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:56.299428 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-psn5f_99b9c479-5323-44a6-bbfa-2aa301efb6af/node-ca/0.log" Jul 13 17:05:56.792127 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:56.792091 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-kzbrg"] Jul 13 17:05:56.816259 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:56.816228 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-kzbrg"] Jul 13 17:05:56.816417 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:56.816335 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-kzbrg" Jul 13 17:05:56.820451 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:56.820433 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 13 17:05:56.820937 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:56.820914 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 13 17:05:56.821044 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:56.820917 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-4vf9n\"" Jul 13 17:05:56.821216 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:56.821200 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 13 17:05:56.821524 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:56.821493 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 13 17:05:56.925609 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:56.925583 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klsfp\" (UniqueName: \"kubernetes.io/projected/724d978b-4365-483d-9fe3-3ecf08716125-kube-api-access-klsfp\") pod \"service-ca-79f68b9f4f-kzbrg\" (UID: \"724d978b-4365-483d-9fe3-3ecf08716125\") " pod="openshift-service-ca/service-ca-79f68b9f4f-kzbrg" Jul 13 17:05:56.925766 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:56.925746 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/724d978b-4365-483d-9fe3-3ecf08716125-signing-key\") pod \"service-ca-79f68b9f4f-kzbrg\" (UID: \"724d978b-4365-483d-9fe3-3ecf08716125\") " pod="openshift-service-ca/service-ca-79f68b9f4f-kzbrg" Jul 13 17:05:56.925838 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:56.925782 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/724d978b-4365-483d-9fe3-3ecf08716125-signing-cabundle\") pod \"service-ca-79f68b9f4f-kzbrg\" (UID: \"724d978b-4365-483d-9fe3-3ecf08716125\") " pod="openshift-service-ca/service-ca-79f68b9f4f-kzbrg" Jul 13 17:05:57.026853 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:57.026827 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/724d978b-4365-483d-9fe3-3ecf08716125-signing-key\") pod \"service-ca-79f68b9f4f-kzbrg\" (UID: \"724d978b-4365-483d-9fe3-3ecf08716125\") " pod="openshift-service-ca/service-ca-79f68b9f4f-kzbrg" Jul 13 17:05:57.026948 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:57.026858 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/724d978b-4365-483d-9fe3-3ecf08716125-signing-cabundle\") pod \"service-ca-79f68b9f4f-kzbrg\" (UID: \"724d978b-4365-483d-9fe3-3ecf08716125\") " pod="openshift-service-ca/service-ca-79f68b9f4f-kzbrg" Jul 13 17:05:57.026994 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:57.026956 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klsfp\" (UniqueName: \"kubernetes.io/projected/724d978b-4365-483d-9fe3-3ecf08716125-kube-api-access-klsfp\") pod \"service-ca-79f68b9f4f-kzbrg\" (UID: \"724d978b-4365-483d-9fe3-3ecf08716125\") " pod="openshift-service-ca/service-ca-79f68b9f4f-kzbrg" Jul 13 17:05:57.027557 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:57.027534 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/724d978b-4365-483d-9fe3-3ecf08716125-signing-cabundle\") pod \"service-ca-79f68b9f4f-kzbrg\" (UID: \"724d978b-4365-483d-9fe3-3ecf08716125\") " pod="openshift-service-ca/service-ca-79f68b9f4f-kzbrg" Jul 13 17:05:57.028858 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:57.028838 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/724d978b-4365-483d-9fe3-3ecf08716125-signing-key\") pod \"service-ca-79f68b9f4f-kzbrg\" (UID: \"724d978b-4365-483d-9fe3-3ecf08716125\") " pod="openshift-service-ca/service-ca-79f68b9f4f-kzbrg" Jul 13 17:05:57.036442 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:57.036421 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-klsfp\" (UniqueName: \"kubernetes.io/projected/724d978b-4365-483d-9fe3-3ecf08716125-kube-api-access-klsfp\") pod \"service-ca-79f68b9f4f-kzbrg\" (UID: \"724d978b-4365-483d-9fe3-3ecf08716125\") " pod="openshift-service-ca/service-ca-79f68b9f4f-kzbrg" Jul 13 17:05:57.099605 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:57.099545 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5s55t_2f125df3-6cb8-4ec4-bd2f-24428ce6127e/serve-healthcheck-canary/0.log" Jul 13 17:05:57.127358 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:57.127333 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-kzbrg" Jul 13 17:05:57.254722 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:57.254685 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-kzbrg"] Jul 13 17:05:57.257245 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:05:57.257216 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod724d978b_4365_483d_9fe3_3ecf08716125.slice/crio-8e449955d58fd8d05a51db063dac22052e40bf6bebe97c38318f70e193dcb18b WatchSource:0}: Error finding container 8e449955d58fd8d05a51db063dac22052e40bf6bebe97c38318f70e193dcb18b: Status 404 returned error can't find the container with id 8e449955d58fd8d05a51db063dac22052e40bf6bebe97c38318f70e193dcb18b Jul 13 17:05:57.558488 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:57.558454 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-9pnpw" event={"ID":"4e92fb26-ea9b-4a99-bc67-5e52b023aa20","Type":"ContainerStarted","Data":"e40ae53a78e83d8036e32c15f4756b10fb853cacbdf4894a4e51385bd6ce3628"} Jul 13 17:05:57.558658 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:57.558522 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-9pnpw" event={"ID":"4e92fb26-ea9b-4a99-bc67-5e52b023aa20","Type":"ContainerStarted","Data":"60b9bb999d171ce30da7f99ca5dc29bd9c7db64fe98ed0c9d3724c33693e7e09"} Jul 13 17:05:57.559916 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:57.559894 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-kzbrg" event={"ID":"724d978b-4365-483d-9fe3-3ecf08716125","Type":"ContainerStarted","Data":"1c25158b6057895d7e394aa663a0f38f351a8b0164986c617518d2117f167867"} Jul 13 17:05:57.560039 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:57.559938 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-kzbrg" event={"ID":"724d978b-4365-483d-9fe3-3ecf08716125","Type":"ContainerStarted","Data":"8e449955d58fd8d05a51db063dac22052e40bf6bebe97c38318f70e193dcb18b"} Jul 13 17:05:57.576526 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:57.576479 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-9pnpw" podStartSLOduration=1.915277846 podStartE2EDuration="3.576465122s" totalImagesPullingTime="1.661187276s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:05:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:05:57.574927829 +0000 UTC m=+53.972586207" watchObservedRunningTime="2026-07-13 17:05:57.576465122 +0000 UTC m=+53.974123500" Jul 13 17:05:57.599405 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:57.599363 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-kzbrg" podStartSLOduration=1.599351929 podStartE2EDuration="1.599351929s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:05:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:05:57.598765212 +0000 UTC m=+53.996423591" watchObservedRunningTime="2026-07-13 17:05:57.599351929 +0000 UTC m=+53.997010307" Jul 13 17:05:57.853622 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:57.853536 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-hzcql" Jul 13 17:05:57.853622 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:57.853613 2569 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-hzcql" Jul 13 17:05:57.854001 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:57.853988 2569 scope.go:117] "RemoveContainer" containerID="00e88729dcc709480a17df78650156a21d942845bb31323ac69c79ca9b0fe642" Jul 13 17:05:57.854178 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:57.854156 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-hzcql_openshift-console-operator(c40e6e88-ab3f-4e3c-a0e7-2400bda333eb)\"" pod="openshift-console-operator/console-operator-7866497494-hzcql" podUID="c40e6e88-ab3f-4e3c-a0e7-2400bda333eb" Jul 13 17:05:58.035921 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:58.035891 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/965ad197-c001-48c3-9ffd-58b4d7415d1e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cfcv4\" (UID: \"965ad197-c001-48c3-9ffd-58b4d7415d1e\") " pod="openshift-insights/insights-runtime-extractor-cfcv4" Jul 13 17:05:58.036069 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:05:58.036002 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/965ad197-c001-48c3-9ffd-58b4d7415d1e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cfcv4\" (UID: \"965ad197-c001-48c3-9ffd-58b4d7415d1e\") " pod="openshift-insights/insights-runtime-extractor-cfcv4" Jul 13 17:05:58.036069 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:58.036031 2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 17:05:58.036170 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:58.036097 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/965ad197-c001-48c3-9ffd-58b4d7415d1e-insights-runtime-extractor-tls podName:965ad197-c001-48c3-9ffd-58b4d7415d1e nodeName:}" failed. No retries permitted until 2026-07-13 17:06:02.036078222 +0000 UTC m=+58.433736593 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/965ad197-c001-48c3-9ffd-58b4d7415d1e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cfcv4" (UID: "965ad197-c001-48c3-9ffd-58b4d7415d1e") : secret "insights-runtime-extractor-tls" not found Jul 13 17:05:58.036228 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:58.036172 2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 13 17:05:58.036281 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:05:58.036234 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/965ad197-c001-48c3-9ffd-58b4d7415d1e-kube-rbac-proxy-cm podName:965ad197-c001-48c3-9ffd-58b4d7415d1e nodeName:}" failed. No retries permitted until 2026-07-13 17:06:02.036217338 +0000 UTC m=+58.433875699 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/965ad197-c001-48c3-9ffd-58b4d7415d1e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cfcv4" (UID: "965ad197-c001-48c3-9ffd-58b4d7415d1e") : configmap "kube-rbac-proxy" not found Jul 13 17:06:02.073055 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:02.073016 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/965ad197-c001-48c3-9ffd-58b4d7415d1e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cfcv4\" (UID: \"965ad197-c001-48c3-9ffd-58b4d7415d1e\") " pod="openshift-insights/insights-runtime-extractor-cfcv4" Jul 13 17:06:02.073635 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:02.073080 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/965ad197-c001-48c3-9ffd-58b4d7415d1e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cfcv4\" (UID: \"965ad197-c001-48c3-9ffd-58b4d7415d1e\") " pod="openshift-insights/insights-runtime-extractor-cfcv4" Jul 13 17:06:02.073635 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:06:02.073167 2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 17:06:02.073635 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:06:02.073207 2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 13 17:06:02.073635 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:06:02.073238 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/965ad197-c001-48c3-9ffd-58b4d7415d1e-insights-runtime-extractor-tls podName:965ad197-c001-48c3-9ffd-58b4d7415d1e nodeName:}" failed. No retries permitted until 2026-07-13 17:06:10.073221154 +0000 UTC m=+66.470879514 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/965ad197-c001-48c3-9ffd-58b4d7415d1e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cfcv4" (UID: "965ad197-c001-48c3-9ffd-58b4d7415d1e") : secret "insights-runtime-extractor-tls" not found Jul 13 17:06:02.073635 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:06:02.073261 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/965ad197-c001-48c3-9ffd-58b4d7415d1e-kube-rbac-proxy-cm podName:965ad197-c001-48c3-9ffd-58b4d7415d1e nodeName:}" failed. No retries permitted until 2026-07-13 17:06:10.073248205 +0000 UTC m=+66.470906565 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/965ad197-c001-48c3-9ffd-58b4d7415d1e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cfcv4" (UID: "965ad197-c001-48c3-9ffd-58b4d7415d1e") : configmap "kube-rbac-proxy" not found Jul 13 17:06:03.288669 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:03.288643 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-q2pnb" Jul 13 17:06:09.338709 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:09.338674 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/46bae189-2ca3-416c-bcb7-42b1283f71b8-registry-tls\") pod \"image-registry-5878cf7589-sgggs\" (UID: \"46bae189-2ca3-416c-bcb7-42b1283f71b8\") " pod="openshift-image-registry/image-registry-5878cf7589-sgggs" Jul 13 17:06:09.339225 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:09.338727 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1a2cb5a4-a8a6-4519-b001-9606305fc3a1-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-lzd48\" (UID: \"1a2cb5a4-a8a6-4519-b001-9606305fc3a1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-lzd48" Jul 13 17:06:09.339225 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:09.338788 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7549ca91-bd0b-4b2f-b76c-0a7784f2fd4c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-ttd4d\" (UID: \"7549ca91-bd0b-4b2f-b76c-0a7784f2fd4c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-ttd4d" Jul 13 17:06:09.341202 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:09.341167 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/46bae189-2ca3-416c-bcb7-42b1283f71b8-registry-tls\") pod \"image-registry-5878cf7589-sgggs\" (UID: \"46bae189-2ca3-416c-bcb7-42b1283f71b8\") " pod="openshift-image-registry/image-registry-5878cf7589-sgggs" Jul 13 17:06:09.341303 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:09.341207 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1a2cb5a4-a8a6-4519-b001-9606305fc3a1-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-lzd48\" (UID: \"1a2cb5a4-a8a6-4519-b001-9606305fc3a1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-lzd48" Jul 13 17:06:09.341352 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:09.341302 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7549ca91-bd0b-4b2f-b76c-0a7784f2fd4c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-ttd4d\" (UID: \"7549ca91-bd0b-4b2f-b76c-0a7784f2fd4c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-ttd4d" Jul 13 17:06:09.365174 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:09.365155 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-425xq\"" Jul 13 17:06:09.373631 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:09.373608 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-lzd48" Jul 13 17:06:09.439990 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:09.439962 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e443fc05-8996-474b-85fa-f3a28463785a-service-ca-bundle\") pod \"router-default-969d568db-zxb2r\" (UID: \"e443fc05-8996-474b-85fa-f3a28463785a\") " pod="openshift-ingress/router-default-969d568db-zxb2r" Jul 13 17:06:09.440165 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:09.440019 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e443fc05-8996-474b-85fa-f3a28463785a-metrics-certs\") pod \"router-default-969d568db-zxb2r\" (UID: \"e443fc05-8996-474b-85fa-f3a28463785a\") " pod="openshift-ingress/router-default-969d568db-zxb2r" Jul 13 17:06:09.440165 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:09.440060 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/afb76ab4-2a66-45a7-8c98-7c3ca6e026b9-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-5wvjq\" (UID: \"afb76ab4-2a66-45a7-8c98-7c3ca6e026b9\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-5wvjq" Jul 13 17:06:09.440165 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:09.440095 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/57b96b57-d3e5-44f3-bcca-2a077820b845-metrics-tls\") pod \"dns-default-9xwzd\" (UID: \"57b96b57-d3e5-44f3-bcca-2a077820b845\") " pod="openshift-dns/dns-default-9xwzd" Jul 13 17:06:09.441005 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:09.440957 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e443fc05-8996-474b-85fa-f3a28463785a-service-ca-bundle\") pod \"router-default-969d568db-zxb2r\" (UID: \"e443fc05-8996-474b-85fa-f3a28463785a\") " pod="openshift-ingress/router-default-969d568db-zxb2r" Jul 13 17:06:09.443111 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:09.443063 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/57b96b57-d3e5-44f3-bcca-2a077820b845-metrics-tls\") pod \"dns-default-9xwzd\" (UID: \"57b96b57-d3e5-44f3-bcca-2a077820b845\") " pod="openshift-dns/dns-default-9xwzd" Jul 13 17:06:09.443234 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:09.443210 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/afb76ab4-2a66-45a7-8c98-7c3ca6e026b9-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-5wvjq\" (UID: \"afb76ab4-2a66-45a7-8c98-7c3ca6e026b9\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-5wvjq" Jul 13 17:06:09.443665 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:09.443633 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e443fc05-8996-474b-85fa-f3a28463785a-metrics-certs\") pod \"router-default-969d568db-zxb2r\" (UID: \"e443fc05-8996-474b-85fa-f3a28463785a\") " pod="openshift-ingress/router-default-969d568db-zxb2r" Jul 13 17:06:09.499332 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:09.499302 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-lzd48"] Jul 13 17:06:09.602097 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:09.602046 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-lzd48" event={"ID":"1a2cb5a4-a8a6-4519-b001-9606305fc3a1","Type":"ContainerStarted","Data":"0c7cb267037fc2c5091d4aefef8cc466ff8c1395aad8816fee963790bfc83cea"} Jul 13 17:06:09.603328 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:09.603313 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-ctbft\"" Jul 13 17:06:09.611025 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:09.611009 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5878cf7589-sgggs" Jul 13 17:06:09.612889 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:09.612876 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-w4bnh\"" Jul 13 17:06:09.621611 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:09.621593 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-ttd4d" Jul 13 17:06:09.681808 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:09.681777 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-wbxl8\"" Jul 13 17:06:09.688746 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:09.688722 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xtcxn\"" Jul 13 17:06:09.689452 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:09.689431 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-969d568db-zxb2r" Jul 13 17:06:09.696816 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:09.696790 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9xwzd" Jul 13 17:06:09.701920 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:09.701903 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-s7vl4\"" Jul 13 17:06:09.710576 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:09.710542 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-5wvjq" Jul 13 17:06:09.751815 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:09.751494 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5878cf7589-sgggs"] Jul 13 17:06:09.754978 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:06:09.754945 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46bae189_2ca3_416c_bcb7_42b1283f71b8.slice/crio-f0389d12057b1b59150b4ce39210a8fa45f6b14bb7e7ecc99812ca19359e3069 WatchSource:0}: Error finding container f0389d12057b1b59150b4ce39210a8fa45f6b14bb7e7ecc99812ca19359e3069: Status 404 returned error can't find the container with id f0389d12057b1b59150b4ce39210a8fa45f6b14bb7e7ecc99812ca19359e3069 Jul 13 17:06:09.773677 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:06:09.773632 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7549ca91_bd0b_4b2f_b76c_0a7784f2fd4c.slice/crio-3163c4f003cbb2ed710fb816a8edb771733f131a5d1986f51b4530098010478f WatchSource:0}: Error finding container 3163c4f003cbb2ed710fb816a8edb771733f131a5d1986f51b4530098010478f: Status 404 returned error can't find the container with id 3163c4f003cbb2ed710fb816a8edb771733f131a5d1986f51b4530098010478f Jul 13 17:06:09.779825 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:09.777017 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-ttd4d"] Jul 13 17:06:09.842977 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:09.842756 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7cb8e14a-393f-43f2-b75a-7f14756cfb2a-metrics-certs\") pod \"network-metrics-daemon-t6r98\" (UID: \"7cb8e14a-393f-43f2-b75a-7f14756cfb2a\") " pod="openshift-multus/network-metrics-daemon-t6r98" Jul 13 17:06:09.842977 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:09.842825 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2263bf98-5f88-41d9-b755-9f874a3d5916-original-pull-secret\") pod \"global-pull-secret-syncer-nfjpb\" (UID: \"2263bf98-5f88-41d9-b755-9f874a3d5916\") " pod="kube-system/global-pull-secret-syncer-nfjpb" Jul 13 17:06:09.846266 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:09.846243 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 13 17:06:09.846266 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:09.846253 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 13 17:06:09.856125 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:09.856045 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7cb8e14a-393f-43f2-b75a-7f14756cfb2a-metrics-certs\") pod \"network-metrics-daemon-t6r98\" (UID: \"7cb8e14a-393f-43f2-b75a-7f14756cfb2a\") " pod="openshift-multus/network-metrics-daemon-t6r98" Jul 13 17:06:09.857271 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:09.857245 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2263bf98-5f88-41d9-b755-9f874a3d5916-original-pull-secret\") pod \"global-pull-secret-syncer-nfjpb\" (UID: \"2263bf98-5f88-41d9-b755-9f874a3d5916\") " pod="kube-system/global-pull-secret-syncer-nfjpb" Jul 13 17:06:09.888260 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:06:09.888233 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57b96b57_d3e5_44f3_bcca_2a077820b845.slice/crio-36426ee7e677a709f64618219f6e51c49042180358bebf22ef54ebc57f801ec1 WatchSource:0}: Error finding container 36426ee7e677a709f64618219f6e51c49042180358bebf22ef54ebc57f801ec1: Status 404 returned error can't find the container with id 36426ee7e677a709f64618219f6e51c49042180358bebf22ef54ebc57f801ec1 Jul 13 17:06:09.893176 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:09.893148 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9xwzd"] Jul 13 17:06:09.905692 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:09.905669 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-969d568db-zxb2r"] Jul 13 17:06:09.908706 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:06:09.908681 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode443fc05_8996_474b_85fa_f3a28463785a.slice/crio-9a7770b389a05e2e0ad94e5d2be96e64fc6287e132ed797f07eb5a9ecab7d394 WatchSource:0}: Error finding container 9a7770b389a05e2e0ad94e5d2be96e64fc6287e132ed797f07eb5a9ecab7d394: Status 404 returned error can't find the container with id 9a7770b389a05e2e0ad94e5d2be96e64fc6287e132ed797f07eb5a9ecab7d394 Jul 13 17:06:09.924328 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:09.924303 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-5wvjq"] Jul 13 17:06:09.926862 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:06:09.926838 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafb76ab4_2a66_45a7_8c98_7c3ca6e026b9.slice/crio-ec30c740741618059b2fa7aa8fcd2a195ce816eea4427f3bc89c95920c4e5429 WatchSource:0}: Error finding container ec30c740741618059b2fa7aa8fcd2a195ce816eea4427f3bc89c95920c4e5429: Status 404 returned error can't find the container with id ec30c740741618059b2fa7aa8fcd2a195ce816eea4427f3bc89c95920c4e5429 Jul 13 17:06:09.957073 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:09.957046 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nfjpb" Jul 13 17:06:10.052014 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:10.051990 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5r2pv\"" Jul 13 17:06:10.060512 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:10.060486 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6r98" Jul 13 17:06:10.112222 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:10.112101 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-nfjpb"] Jul 13 17:06:10.115620 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:06:10.115592 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2263bf98_5f88_41d9_b755_9f874a3d5916.slice/crio-4f0ac032503a60923f5c7a2625fcd96e521b8d7985a4445a17b42a906690c413 WatchSource:0}: Error finding container 4f0ac032503a60923f5c7a2625fcd96e521b8d7985a4445a17b42a906690c413: Status 404 returned error can't find the container with id 4f0ac032503a60923f5c7a2625fcd96e521b8d7985a4445a17b42a906690c413 Jul 13 17:06:10.146228 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:10.145941 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/965ad197-c001-48c3-9ffd-58b4d7415d1e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cfcv4\" (UID: \"965ad197-c001-48c3-9ffd-58b4d7415d1e\") " pod="openshift-insights/insights-runtime-extractor-cfcv4" Jul 13 17:06:10.146228 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:10.146049 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/965ad197-c001-48c3-9ffd-58b4d7415d1e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cfcv4\" (UID: \"965ad197-c001-48c3-9ffd-58b4d7415d1e\") " pod="openshift-insights/insights-runtime-extractor-cfcv4" Jul 13 17:06:10.146687 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:10.146665 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/965ad197-c001-48c3-9ffd-58b4d7415d1e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cfcv4\" (UID: \"965ad197-c001-48c3-9ffd-58b4d7415d1e\") " pod="openshift-insights/insights-runtime-extractor-cfcv4" Jul 13 17:06:10.148555 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:10.148530 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/965ad197-c001-48c3-9ffd-58b4d7415d1e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cfcv4\" (UID: \"965ad197-c001-48c3-9ffd-58b4d7415d1e\") " pod="openshift-insights/insights-runtime-extractor-cfcv4" Jul 13 17:06:10.208278 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:10.208244 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-t6r98"] Jul 13 17:06:10.211546 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:06:10.211519 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cb8e14a_393f_43f2_b75a_7f14756cfb2a.slice/crio-448a6d27b90c56c38dde6377ff87937a348d55a9c0fdf2a11f039fd496d5d009 WatchSource:0}: Error finding container 448a6d27b90c56c38dde6377ff87937a348d55a9c0fdf2a11f039fd496d5d009: Status 404 returned error can't find the container with id 448a6d27b90c56c38dde6377ff87937a348d55a9c0fdf2a11f039fd496d5d009 Jul 13 17:06:10.224377 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:10.224357 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rmd8z\"" Jul 13 17:06:10.231989 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:10.231971 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cfcv4" Jul 13 17:06:10.404520 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:10.404064 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-cfcv4"] Jul 13 17:06:10.407526 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:06:10.407475 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod965ad197_c001_48c3_9ffd_58b4d7415d1e.slice/crio-1a6bd46d7015f2a006593042bd1b38dc401c3312c1428cfba0895528082e1219 WatchSource:0}: Error finding container 1a6bd46d7015f2a006593042bd1b38dc401c3312c1428cfba0895528082e1219: Status 404 returned error can't find the container with id 1a6bd46d7015f2a006593042bd1b38dc401c3312c1428cfba0895528082e1219 Jul 13 17:06:10.608786 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:10.608751 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5878cf7589-sgggs" event={"ID":"46bae189-2ca3-416c-bcb7-42b1283f71b8","Type":"ContainerStarted","Data":"72e8ecfcb766c4a1881dc51c2909f45cbf1d5bf35cca9fe01156f60260b3af42"} Jul 13 17:06:10.608965 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:10.608810 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5878cf7589-sgggs" event={"ID":"46bae189-2ca3-416c-bcb7-42b1283f71b8","Type":"ContainerStarted","Data":"f0389d12057b1b59150b4ce39210a8fa45f6b14bb7e7ecc99812ca19359e3069"} Jul 13 17:06:10.608965 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:10.608892 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5878cf7589-sgggs" Jul 13 17:06:10.610279 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:10.610254 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-nfjpb" event={"ID":"2263bf98-5f88-41d9-b755-9f874a3d5916","Type":"ContainerStarted","Data":"4f0ac032503a60923f5c7a2625fcd96e521b8d7985a4445a17b42a906690c413"} Jul 13 17:06:10.611854 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:10.611832 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-ttd4d" event={"ID":"7549ca91-bd0b-4b2f-b76c-0a7784f2fd4c","Type":"ContainerStarted","Data":"3163c4f003cbb2ed710fb816a8edb771733f131a5d1986f51b4530098010478f"} Jul 13 17:06:10.613467 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:10.613433 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9xwzd" event={"ID":"57b96b57-d3e5-44f3-bcca-2a077820b845","Type":"ContainerStarted","Data":"36426ee7e677a709f64618219f6e51c49042180358bebf22ef54ebc57f801ec1"} Jul 13 17:06:10.615126 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:10.615094 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cfcv4" event={"ID":"965ad197-c001-48c3-9ffd-58b4d7415d1e","Type":"ContainerStarted","Data":"1a6bd46d7015f2a006593042bd1b38dc401c3312c1428cfba0895528082e1219"} Jul 13 17:06:10.616651 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:10.616590 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-t6r98" event={"ID":"7cb8e14a-393f-43f2-b75a-7f14756cfb2a","Type":"ContainerStarted","Data":"448a6d27b90c56c38dde6377ff87937a348d55a9c0fdf2a11f039fd496d5d009"} Jul 13 17:06:10.617980 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:10.617958 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-5wvjq" event={"ID":"afb76ab4-2a66-45a7-8c98-7c3ca6e026b9","Type":"ContainerStarted","Data":"ec30c740741618059b2fa7aa8fcd2a195ce816eea4427f3bc89c95920c4e5429"} Jul 13 17:06:10.619973 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:10.619945 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-969d568db-zxb2r" event={"ID":"e443fc05-8996-474b-85fa-f3a28463785a","Type":"ContainerStarted","Data":"d23037f73a0a657125b7db390a15068695a0ae482f258a85d55db5c44f301074"} Jul 13 17:06:10.620065 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:10.619983 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-969d568db-zxb2r" event={"ID":"e443fc05-8996-474b-85fa-f3a28463785a","Type":"ContainerStarted","Data":"9a7770b389a05e2e0ad94e5d2be96e64fc6287e132ed797f07eb5a9ecab7d394"} Jul 13 17:06:10.631985 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:10.631936 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5878cf7589-sgggs" podStartSLOduration=66.63191908 podStartE2EDuration="1m6.63191908s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:05:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:06:10.629779058 +0000 UTC m=+67.027437438" watchObservedRunningTime="2026-07-13 17:06:10.63191908 +0000 UTC m=+67.029577462" Jul 13 17:06:10.653472 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:10.653384 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-969d568db-zxb2r" podStartSLOduration=46.65336579 podStartE2EDuration="46.65336579s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:05:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:06:10.651859044 +0000 UTC m=+67.049517422" watchObservedRunningTime="2026-07-13 17:06:10.65336579 +0000 UTC m=+67.051024169" Jul 13 17:06:10.689894 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:10.689820 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-969d568db-zxb2r" Jul 13 17:06:10.692845 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:10.692821 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-969d568db-zxb2r" Jul 13 17:06:11.632163 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:11.631871 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-969d568db-zxb2r" Jul 13 17:06:11.633185 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:11.632915 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-969d568db-zxb2r" Jul 13 17:06:12.132328 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:12.131998 2569 scope.go:117] "RemoveContainer" containerID="00e88729dcc709480a17df78650156a21d942845bb31323ac69c79ca9b0fe642" Jul 13 17:06:17.672610 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:17.672287 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-t6r98" event={"ID":"7cb8e14a-393f-43f2-b75a-7f14756cfb2a","Type":"ContainerStarted","Data":"5790afa9b380cf77cd348dd0ca2a0f09d9f6b300f2bc9c73b20ee284e3be73fa"} Jul 13 17:06:17.679244 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:17.678375 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-5wvjq" event={"ID":"afb76ab4-2a66-45a7-8c98-7c3ca6e026b9","Type":"ContainerStarted","Data":"792a3d202cb41828ae3a410cf06233c5d67be927560628b942ce9e4a1104c867"} Jul 13 17:06:17.683132 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:17.682951 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-hzcql_c40e6e88-ab3f-4e3c-a0e7-2400bda333eb/console-operator/1.log" Jul 13 17:06:17.683132 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:17.683054 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-hzcql" event={"ID":"c40e6e88-ab3f-4e3c-a0e7-2400bda333eb","Type":"ContainerStarted","Data":"5be3333d501af12659f560267a7b8df03b50e9ede9047fdb48af392152da5848"} Jul 13 17:06:17.684893 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:17.684823 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-ttd4d" event={"ID":"7549ca91-bd0b-4b2f-b76c-0a7784f2fd4c","Type":"ContainerStarted","Data":"27f13298604b2d0c820bdf36570c8d05070e547aa58f978daff8b9202746d290"} Jul 13 17:06:17.688331 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:17.688300 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-lzd48" event={"ID":"1a2cb5a4-a8a6-4519-b001-9606305fc3a1","Type":"ContainerStarted","Data":"5b08f16434c5e4ab28dc8643fc6d486bee0170fd170ce2ae84b35c8e913a4994"} Jul 13 17:06:17.692866 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:17.692817 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-hzcql" Jul 13 17:06:17.699243 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:17.699200 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-hzcql" Jul 13 17:06:17.724303 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:17.723777 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-ttd4d" podStartSLOduration=46.378273118 podStartE2EDuration="53.723757165s" totalImagesPullingTime="7.345484047s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:05:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:06:17.722675648 +0000 UTC m=+74.120334045" watchObservedRunningTime="2026-07-13 17:06:17.723757165 +0000 UTC m=+74.121415548" Jul 13 17:06:17.764824 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:17.764768 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-hzcql" podStartSLOduration=40.022449727 podStartE2EDuration="53.764745204s" totalImagesPullingTime="13.742295477s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:05:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:06:17.763415565 +0000 UTC m=+74.161073946" watchObservedRunningTime="2026-07-13 17:06:17.764745204 +0000 UTC m=+74.162403586" Jul 13 17:06:17.840835 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:17.840522 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-5wvjq" podStartSLOduration=46.647830875 podStartE2EDuration="53.840498039s" totalImagesPullingTime="7.192667164s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:05:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:06:17.837335684 +0000 UTC m=+74.234994076" watchObservedRunningTime="2026-07-13 17:06:17.840498039 +0000 UTC m=+74.238156419" Jul 13 17:06:18.694683 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:18.694628 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-t6r98" event={"ID":"7cb8e14a-393f-43f2-b75a-7f14756cfb2a","Type":"ContainerStarted","Data":"8afb1b830183dd889cccf3891f9aa9dab49e1bef915a3f6d775d68fedb2b289e"} Jul 13 17:06:18.696415 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:18.696374 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-nfjpb" event={"ID":"2263bf98-5f88-41d9-b755-9f874a3d5916","Type":"ContainerStarted","Data":"7bef48977a15e59024796c0c4410c05072d29cd9fa59a4fe4b952315c515b5bc"} Jul 13 17:06:18.698547 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:18.698521 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9xwzd" event={"ID":"57b96b57-d3e5-44f3-bcca-2a077820b845","Type":"ContainerStarted","Data":"8d399d99680338033087adff0dc2c0077ba8cc01dc831134e76e503e1df7482f"} Jul 13 17:06:18.698686 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:18.698589 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9xwzd" event={"ID":"57b96b57-d3e5-44f3-bcca-2a077820b845","Type":"ContainerStarted","Data":"8788fb5656a66e4f7f6f19c9545a73b7bd4d5f48a6fbb08d3b7387277a0b110b"} Jul 13 17:06:18.698686 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:18.698620 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-9xwzd" Jul 13 17:06:18.700798 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:18.700773 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-lzd48" event={"ID":"1a2cb5a4-a8a6-4519-b001-9606305fc3a1","Type":"ContainerStarted","Data":"926c5bd85c0ea198accd0d01a3a02a4e455e72a9672bdad00bcdc397dfb2a964"} Jul 13 17:06:18.702511 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:18.702489 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cfcv4" event={"ID":"965ad197-c001-48c3-9ffd-58b4d7415d1e","Type":"ContainerStarted","Data":"95b52c850e7532f7dbe04c5c9c39aaf1c6e45d801c10b64e71a6201aec22ad1e"} Jul 13 17:06:18.713953 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:18.713903 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-t6r98" podStartSLOduration=68.404467173 podStartE2EDuration="1m14.713885645s" totalImagesPullingTime="6.309418472s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:05:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:06:18.711712027 +0000 UTC m=+75.109370405" watchObservedRunningTime="2026-07-13 17:06:18.713885645 +0000 UTC m=+75.111544025" Jul 13 17:06:18.728963 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:18.728919 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-9xwzd" podStartSLOduration=34.135692272 podStartE2EDuration="41.728906564s" totalImagesPullingTime="7.593214292s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:05:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:06:18.727373928 +0000 UTC m=+75.125032320" watchObservedRunningTime="2026-07-13 17:06:18.728906564 +0000 UTC m=+75.126564943" Jul 13 17:06:18.743795 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:18.743751 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-nfjpb" podStartSLOduration=67.376585135 podStartE2EDuration="1m14.74373793s" totalImagesPullingTime="7.367152795s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 17:05:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:06:18.742675413 +0000 UTC m=+75.140333793" watchObservedRunningTime="2026-07-13 17:06:18.74373793 +0000 UTC m=+75.141396302" Jul 13 17:06:18.759312 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:18.759273 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-lzd48" podStartSLOduration=47.183060425 podStartE2EDuration="54.759261629s" totalImagesPullingTime="7.576201204s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:05:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:06:18.758371749 +0000 UTC m=+75.156030148" watchObservedRunningTime="2026-07-13 17:06:18.759261629 +0000 UTC m=+75.156920008" Jul 13 17:06:19.710080 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:19.709996 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cfcv4" event={"ID":"965ad197-c001-48c3-9ffd-58b4d7415d1e","Type":"ContainerStarted","Data":"d01bb14c0bc5df2a75a282983f122784b9f33c9ca431da0b4ed66bbb24e898e7"} Jul 13 17:06:22.726110 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:22.726034 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cfcv4" event={"ID":"965ad197-c001-48c3-9ffd-58b4d7415d1e","Type":"ContainerStarted","Data":"ab771f5754347f28cc3d0f138792b20ebc9c64a19f1e6f6bc7688940df763053"} Jul 13 17:06:22.744531 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:22.744480 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-cfcv4" podStartSLOduration=16.905509946 podStartE2EDuration="28.744464086s" totalImagesPullingTime="11.83895414s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 17:05:54 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:06:22.74335831 +0000 UTC m=+79.141016688" watchObservedRunningTime="2026-07-13 17:06:22.744464086 +0000 UTC m=+79.142122465" Jul 13 17:06:24.545245 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:24.545212 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-2b6wx" Jul 13 17:06:26.750808 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:26.750682 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-qrt8r"] Jul 13 17:06:26.765903 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:26.765876 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qrt8r" Jul 13 17:06:26.767882 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:26.767861 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 13 17:06:26.768254 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:26.768068 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 13 17:06:26.768254 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:26.768095 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 13 17:06:26.768254 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:26.768125 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-tbg8b\"" Jul 13 17:06:26.768254 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:26.768197 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 13 17:06:26.879990 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:26.879962 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ltkg\" (UniqueName: \"kubernetes.io/projected/41a0e37d-a408-4cf1-946d-d573d2d5cfa4-kube-api-access-9ltkg\") pod \"node-exporter-qrt8r\" (UID: \"41a0e37d-a408-4cf1-946d-d573d2d5cfa4\") " pod="openshift-monitoring/node-exporter-qrt8r" Jul 13 17:06:26.879990 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:26.879996 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/41a0e37d-a408-4cf1-946d-d573d2d5cfa4-metrics-client-ca\") pod \"node-exporter-qrt8r\" (UID: \"41a0e37d-a408-4cf1-946d-d573d2d5cfa4\") " pod="openshift-monitoring/node-exporter-qrt8r" Jul 13 17:06:26.880206 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:26.880015 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/41a0e37d-a408-4cf1-946d-d573d2d5cfa4-node-exporter-wtmp\") pod \"node-exporter-qrt8r\" (UID: \"41a0e37d-a408-4cf1-946d-d573d2d5cfa4\") " pod="openshift-monitoring/node-exporter-qrt8r" Jul 13 17:06:26.880206 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:26.880074 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/41a0e37d-a408-4cf1-946d-d573d2d5cfa4-root\") pod \"node-exporter-qrt8r\" (UID: \"41a0e37d-a408-4cf1-946d-d573d2d5cfa4\") " pod="openshift-monitoring/node-exporter-qrt8r" Jul 13 17:06:26.880206 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:26.880117 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/41a0e37d-a408-4cf1-946d-d573d2d5cfa4-node-exporter-tls\") pod \"node-exporter-qrt8r\" (UID: \"41a0e37d-a408-4cf1-946d-d573d2d5cfa4\") " pod="openshift-monitoring/node-exporter-qrt8r" Jul 13 17:06:26.880206 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:26.880145 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/41a0e37d-a408-4cf1-946d-d573d2d5cfa4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qrt8r\" (UID: \"41a0e37d-a408-4cf1-946d-d573d2d5cfa4\") " pod="openshift-monitoring/node-exporter-qrt8r" Jul 13 17:06:26.880206 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:26.880189 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/41a0e37d-a408-4cf1-946d-d573d2d5cfa4-sys\") pod \"node-exporter-qrt8r\" (UID: \"41a0e37d-a408-4cf1-946d-d573d2d5cfa4\") " pod="openshift-monitoring/node-exporter-qrt8r" Jul 13 17:06:26.880451 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:26.880234 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/41a0e37d-a408-4cf1-946d-d573d2d5cfa4-node-exporter-accelerators-collector-config\") pod \"node-exporter-qrt8r\" (UID: \"41a0e37d-a408-4cf1-946d-d573d2d5cfa4\") " pod="openshift-monitoring/node-exporter-qrt8r" Jul 13 17:06:26.880451 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:26.880266 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/41a0e37d-a408-4cf1-946d-d573d2d5cfa4-node-exporter-textfile\") pod \"node-exporter-qrt8r\" (UID: \"41a0e37d-a408-4cf1-946d-d573d2d5cfa4\") " pod="openshift-monitoring/node-exporter-qrt8r" Jul 13 17:06:26.981197 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:26.981168 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ltkg\" (UniqueName: \"kubernetes.io/projected/41a0e37d-a408-4cf1-946d-d573d2d5cfa4-kube-api-access-9ltkg\") pod \"node-exporter-qrt8r\" (UID: \"41a0e37d-a408-4cf1-946d-d573d2d5cfa4\") " pod="openshift-monitoring/node-exporter-qrt8r" Jul 13 17:06:26.981304 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:26.981212 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/41a0e37d-a408-4cf1-946d-d573d2d5cfa4-metrics-client-ca\") pod \"node-exporter-qrt8r\" (UID: \"41a0e37d-a408-4cf1-946d-d573d2d5cfa4\") " pod="openshift-monitoring/node-exporter-qrt8r" Jul 13 17:06:26.981353 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:26.981333 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/41a0e37d-a408-4cf1-946d-d573d2d5cfa4-node-exporter-wtmp\") pod \"node-exporter-qrt8r\" (UID: \"41a0e37d-a408-4cf1-946d-d573d2d5cfa4\") " pod="openshift-monitoring/node-exporter-qrt8r" Jul 13 17:06:26.981390 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:26.981379 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/41a0e37d-a408-4cf1-946d-d573d2d5cfa4-root\") pod \"node-exporter-qrt8r\" (UID: \"41a0e37d-a408-4cf1-946d-d573d2d5cfa4\") " pod="openshift-monitoring/node-exporter-qrt8r" Jul 13 17:06:26.981435 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:26.981417 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/41a0e37d-a408-4cf1-946d-d573d2d5cfa4-node-exporter-tls\") pod \"node-exporter-qrt8r\" (UID: \"41a0e37d-a408-4cf1-946d-d573d2d5cfa4\") " pod="openshift-monitoring/node-exporter-qrt8r" Jul 13 17:06:26.981486 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:26.981442 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/41a0e37d-a408-4cf1-946d-d573d2d5cfa4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qrt8r\" (UID: \"41a0e37d-a408-4cf1-946d-d573d2d5cfa4\") " pod="openshift-monitoring/node-exporter-qrt8r" Jul 13 17:06:26.981544 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:26.981494 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/41a0e37d-a408-4cf1-946d-d573d2d5cfa4-sys\") pod \"node-exporter-qrt8r\" (UID: \"41a0e37d-a408-4cf1-946d-d573d2d5cfa4\") " pod="openshift-monitoring/node-exporter-qrt8r" Jul 13 17:06:26.981544 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:26.981512 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/41a0e37d-a408-4cf1-946d-d573d2d5cfa4-node-exporter-wtmp\") pod \"node-exporter-qrt8r\" (UID: \"41a0e37d-a408-4cf1-946d-d573d2d5cfa4\") " pod="openshift-monitoring/node-exporter-qrt8r" Jul 13 17:06:26.981544 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:26.981523 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/41a0e37d-a408-4cf1-946d-d573d2d5cfa4-node-exporter-accelerators-collector-config\") pod \"node-exporter-qrt8r\" (UID: \"41a0e37d-a408-4cf1-946d-d573d2d5cfa4\") " pod="openshift-monitoring/node-exporter-qrt8r" Jul 13 17:06:26.981709 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:26.981537 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/41a0e37d-a408-4cf1-946d-d573d2d5cfa4-root\") pod \"node-exporter-qrt8r\" (UID: \"41a0e37d-a408-4cf1-946d-d573d2d5cfa4\") " pod="openshift-monitoring/node-exporter-qrt8r" Jul 13 17:06:26.981709 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:26.981592 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/41a0e37d-a408-4cf1-946d-d573d2d5cfa4-node-exporter-textfile\") pod \"node-exporter-qrt8r\" (UID: \"41a0e37d-a408-4cf1-946d-d573d2d5cfa4\") " pod="openshift-monitoring/node-exporter-qrt8r" Jul 13 17:06:26.981709 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:26.981601 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/41a0e37d-a408-4cf1-946d-d573d2d5cfa4-sys\") pod \"node-exporter-qrt8r\" (UID: \"41a0e37d-a408-4cf1-946d-d573d2d5cfa4\") " pod="openshift-monitoring/node-exporter-qrt8r" Jul 13 17:06:26.981709 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:06:26.981627 2569 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 13 17:06:26.981709 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:06:26.981696 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41a0e37d-a408-4cf1-946d-d573d2d5cfa4-node-exporter-tls podName:41a0e37d-a408-4cf1-946d-d573d2d5cfa4 nodeName:}" failed. No retries permitted until 2026-07-13 17:06:27.4816777 +0000 UTC m=+83.879336060 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/41a0e37d-a408-4cf1-946d-d573d2d5cfa4-node-exporter-tls") pod "node-exporter-qrt8r" (UID: "41a0e37d-a408-4cf1-946d-d573d2d5cfa4") : secret "node-exporter-tls" not found Jul 13 17:06:26.981962 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:26.981883 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/41a0e37d-a408-4cf1-946d-d573d2d5cfa4-node-exporter-textfile\") pod \"node-exporter-qrt8r\" (UID: \"41a0e37d-a408-4cf1-946d-d573d2d5cfa4\") " pod="openshift-monitoring/node-exporter-qrt8r" Jul 13 17:06:26.981962 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:26.981906 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/41a0e37d-a408-4cf1-946d-d573d2d5cfa4-metrics-client-ca\") pod \"node-exporter-qrt8r\" (UID: \"41a0e37d-a408-4cf1-946d-d573d2d5cfa4\") " pod="openshift-monitoring/node-exporter-qrt8r" Jul 13 17:06:26.982114 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:26.982093 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/41a0e37d-a408-4cf1-946d-d573d2d5cfa4-node-exporter-accelerators-collector-config\") pod \"node-exporter-qrt8r\" (UID: \"41a0e37d-a408-4cf1-946d-d573d2d5cfa4\") " pod="openshift-monitoring/node-exporter-qrt8r" Jul 13 17:06:26.983940 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:26.983920 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/41a0e37d-a408-4cf1-946d-d573d2d5cfa4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qrt8r\" (UID: \"41a0e37d-a408-4cf1-946d-d573d2d5cfa4\") " pod="openshift-monitoring/node-exporter-qrt8r" Jul 13 17:06:26.989064 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:26.989042 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ltkg\" (UniqueName: \"kubernetes.io/projected/41a0e37d-a408-4cf1-946d-d573d2d5cfa4-kube-api-access-9ltkg\") pod \"node-exporter-qrt8r\" (UID: \"41a0e37d-a408-4cf1-946d-d573d2d5cfa4\") " pod="openshift-monitoring/node-exporter-qrt8r" Jul 13 17:06:27.484531 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:27.484491 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/41a0e37d-a408-4cf1-946d-d573d2d5cfa4-node-exporter-tls\") pod \"node-exporter-qrt8r\" (UID: \"41a0e37d-a408-4cf1-946d-d573d2d5cfa4\") " pod="openshift-monitoring/node-exporter-qrt8r" Jul 13 17:06:27.486827 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:27.486806 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/41a0e37d-a408-4cf1-946d-d573d2d5cfa4-node-exporter-tls\") pod \"node-exporter-qrt8r\" (UID: \"41a0e37d-a408-4cf1-946d-d573d2d5cfa4\") " pod="openshift-monitoring/node-exporter-qrt8r" Jul 13 17:06:27.676434 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:27.676402 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qrt8r" Jul 13 17:06:27.707579 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:06:27.707536 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41a0e37d_a408_4cf1_946d_d573d2d5cfa4.slice/crio-9873dfff23f298333267bde9aa0b4d4d232bb2f38acd47c65d1f196759de68b4 WatchSource:0}: Error finding container 9873dfff23f298333267bde9aa0b4d4d232bb2f38acd47c65d1f196759de68b4: Status 404 returned error can't find the container with id 9873dfff23f298333267bde9aa0b4d4d232bb2f38acd47c65d1f196759de68b4 Jul 13 17:06:27.747611 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:27.747517 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qrt8r" event={"ID":"41a0e37d-a408-4cf1-946d-d573d2d5cfa4","Type":"ContainerStarted","Data":"9873dfff23f298333267bde9aa0b4d4d232bb2f38acd47c65d1f196759de68b4"} Jul 13 17:06:28.712576 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:28.712535 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-9xwzd" Jul 13 17:06:29.758525 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:29.758493 2569 generic.go:356] "Generic (PLEG): container finished" podID="41a0e37d-a408-4cf1-946d-d573d2d5cfa4" containerID="848de58da930cb62359e64073f11a4beee6e60b8ec8e4db7495b899cd4d14991" exitCode=0 Jul 13 17:06:29.758886 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:29.758539 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qrt8r" event={"ID":"41a0e37d-a408-4cf1-946d-d573d2d5cfa4","Type":"ContainerDied","Data":"848de58da930cb62359e64073f11a4beee6e60b8ec8e4db7495b899cd4d14991"} Jul 13 17:06:30.764711 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:30.764667 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qrt8r" event={"ID":"41a0e37d-a408-4cf1-946d-d573d2d5cfa4","Type":"ContainerStarted","Data":"8f8511b2aac2014e9f9cebea09058f36ae052446d4e494ac5054e6c3e5c0f866"} Jul 13 17:06:30.765184 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:30.764732 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qrt8r" event={"ID":"41a0e37d-a408-4cf1-946d-d573d2d5cfa4","Type":"ContainerStarted","Data":"ddee7a5bd32ece57c99ae998c5ca197a918bde6cb4ab911e2cbc5a044460089f"} Jul 13 17:06:30.783141 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:30.783093 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-qrt8r" podStartSLOduration=3.353942153 podStartE2EDuration="4.783076734s" totalImagesPullingTime="1.33577432s" totalInitContainerRuntime="93.360261ms" isStatefulPod=true podCreationTimestamp="2026-07-13 17:06:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:06:30.781957576 +0000 UTC m=+87.179615956" watchObservedRunningTime="2026-07-13 17:06:30.783076734 +0000 UTC m=+87.180735147" Jul 13 17:06:31.635282 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:31.635253 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5878cf7589-sgggs" Jul 13 17:06:40.258273 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:06:40.258240 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5878cf7589-sgggs"] Jul 13 17:07:05.279046 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:07:05.278975 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5878cf7589-sgggs" podUID="46bae189-2ca3-416c-bcb7-42b1283f71b8" containerName="registry" containerID="cri-o://72e8ecfcb766c4a1881dc51c2909f45cbf1d5bf35cca9fe01156f60260b3af42" gracePeriod=30 Jul 13 17:07:05.534290 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:07:05.534270 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5878cf7589-sgggs" Jul 13 17:07:05.671433 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:07:05.671391 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/46bae189-2ca3-416c-bcb7-42b1283f71b8-ca-trust-extracted\") pod \"46bae189-2ca3-416c-bcb7-42b1283f71b8\" (UID: \"46bae189-2ca3-416c-bcb7-42b1283f71b8\") " Jul 13 17:07:05.671433 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:07:05.671448 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/46bae189-2ca3-416c-bcb7-42b1283f71b8-registry-tls\") pod \"46bae189-2ca3-416c-bcb7-42b1283f71b8\" (UID: \"46bae189-2ca3-416c-bcb7-42b1283f71b8\") " Jul 13 17:07:05.671679 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:07:05.671490 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/46bae189-2ca3-416c-bcb7-42b1283f71b8-image-registry-private-configuration\") pod \"46bae189-2ca3-416c-bcb7-42b1283f71b8\" (UID: \"46bae189-2ca3-416c-bcb7-42b1283f71b8\") " Jul 13 17:07:05.671679 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:07:05.671510 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/46bae189-2ca3-416c-bcb7-42b1283f71b8-installation-pull-secrets\") pod \"46bae189-2ca3-416c-bcb7-42b1283f71b8\" (UID: \"46bae189-2ca3-416c-bcb7-42b1283f71b8\") " Jul 13 17:07:05.671679 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:07:05.671530 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/46bae189-2ca3-416c-bcb7-42b1283f71b8-trusted-ca\") pod \"46bae189-2ca3-416c-bcb7-42b1283f71b8\" (UID: \"46bae189-2ca3-416c-bcb7-42b1283f71b8\") " Jul 13 17:07:05.671679 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:07:05.671557 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c6j6m\" (UniqueName: \"kubernetes.io/projected/46bae189-2ca3-416c-bcb7-42b1283f71b8-kube-api-access-c6j6m\") pod \"46bae189-2ca3-416c-bcb7-42b1283f71b8\" (UID: \"46bae189-2ca3-416c-bcb7-42b1283f71b8\") " Jul 13 17:07:05.671679 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:07:05.671605 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/46bae189-2ca3-416c-bcb7-42b1283f71b8-registry-certificates\") pod \"46bae189-2ca3-416c-bcb7-42b1283f71b8\" (UID: \"46bae189-2ca3-416c-bcb7-42b1283f71b8\") " Jul 13 17:07:05.671679 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:07:05.671658 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/46bae189-2ca3-416c-bcb7-42b1283f71b8-bound-sa-token\") pod \"46bae189-2ca3-416c-bcb7-42b1283f71b8\" (UID: \"46bae189-2ca3-416c-bcb7-42b1283f71b8\") " Jul 13 17:07:05.672174 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:07:05.672108 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46bae189-2ca3-416c-bcb7-42b1283f71b8-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "46bae189-2ca3-416c-bcb7-42b1283f71b8" (UID: "46bae189-2ca3-416c-bcb7-42b1283f71b8"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:07:05.672174 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:07:05.672124 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46bae189-2ca3-416c-bcb7-42b1283f71b8-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "46bae189-2ca3-416c-bcb7-42b1283f71b8" (UID: "46bae189-2ca3-416c-bcb7-42b1283f71b8"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:07:05.674217 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:07:05.674168 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46bae189-2ca3-416c-bcb7-42b1283f71b8-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "46bae189-2ca3-416c-bcb7-42b1283f71b8" (UID: "46bae189-2ca3-416c-bcb7-42b1283f71b8"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 17:07:05.674329 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:07:05.674242 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46bae189-2ca3-416c-bcb7-42b1283f71b8-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "46bae189-2ca3-416c-bcb7-42b1283f71b8" (UID: "46bae189-2ca3-416c-bcb7-42b1283f71b8"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:07:05.674399 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:07:05.674352 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46bae189-2ca3-416c-bcb7-42b1283f71b8-kube-api-access-c6j6m" (OuterVolumeSpecName: "kube-api-access-c6j6m") pod "46bae189-2ca3-416c-bcb7-42b1283f71b8" (UID: "46bae189-2ca3-416c-bcb7-42b1283f71b8"). InnerVolumeSpecName "kube-api-access-c6j6m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:07:05.674475 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:07:05.674450 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46bae189-2ca3-416c-bcb7-42b1283f71b8-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "46bae189-2ca3-416c-bcb7-42b1283f71b8" (UID: "46bae189-2ca3-416c-bcb7-42b1283f71b8"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 17:07:05.674524 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:07:05.674475 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46bae189-2ca3-416c-bcb7-42b1283f71b8-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "46bae189-2ca3-416c-bcb7-42b1283f71b8" (UID: "46bae189-2ca3-416c-bcb7-42b1283f71b8"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:07:05.679770 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:07:05.679738 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46bae189-2ca3-416c-bcb7-42b1283f71b8-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "46bae189-2ca3-416c-bcb7-42b1283f71b8" (UID: "46bae189-2ca3-416c-bcb7-42b1283f71b8"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:07:05.772480 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:07:05.772457 2569 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/46bae189-2ca3-416c-bcb7-42b1283f71b8-trusted-ca\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:07:05.772480 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:07:05.772479 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c6j6m\" (UniqueName: \"kubernetes.io/projected/46bae189-2ca3-416c-bcb7-42b1283f71b8-kube-api-access-c6j6m\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:07:05.772647 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:07:05.772488 2569 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/46bae189-2ca3-416c-bcb7-42b1283f71b8-registry-certificates\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:07:05.772647 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:07:05.772503 2569 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/46bae189-2ca3-416c-bcb7-42b1283f71b8-bound-sa-token\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:07:05.772647 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:07:05.772515 2569 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/46bae189-2ca3-416c-bcb7-42b1283f71b8-ca-trust-extracted\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:07:05.772647 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:07:05.772523 2569 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/46bae189-2ca3-416c-bcb7-42b1283f71b8-registry-tls\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:07:05.772647 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:07:05.772533 2569 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/46bae189-2ca3-416c-bcb7-42b1283f71b8-image-registry-private-configuration\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:07:05.772647 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:07:05.772542 2569 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/46bae189-2ca3-416c-bcb7-42b1283f71b8-installation-pull-secrets\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:07:05.898854 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:07:05.898774 2569 generic.go:356] "Generic (PLEG): container finished" podID="46bae189-2ca3-416c-bcb7-42b1283f71b8" containerID="72e8ecfcb766c4a1881dc51c2909f45cbf1d5bf35cca9fe01156f60260b3af42" exitCode=0 Jul 13 17:07:05.898854 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:07:05.898815 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5878cf7589-sgggs" Jul 13 17:07:05.898854 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:07:05.898840 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5878cf7589-sgggs" event={"ID":"46bae189-2ca3-416c-bcb7-42b1283f71b8","Type":"ContainerDied","Data":"72e8ecfcb766c4a1881dc51c2909f45cbf1d5bf35cca9fe01156f60260b3af42"} Jul 13 17:07:05.899060 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:07:05.898882 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5878cf7589-sgggs" event={"ID":"46bae189-2ca3-416c-bcb7-42b1283f71b8","Type":"ContainerDied","Data":"f0389d12057b1b59150b4ce39210a8fa45f6b14bb7e7ecc99812ca19359e3069"} Jul 13 17:07:05.899060 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:07:05.898902 2569 scope.go:117] "RemoveContainer" containerID="72e8ecfcb766c4a1881dc51c2909f45cbf1d5bf35cca9fe01156f60260b3af42" Jul 13 17:07:05.907513 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:07:05.907493 2569 scope.go:117] "RemoveContainer" containerID="72e8ecfcb766c4a1881dc51c2909f45cbf1d5bf35cca9fe01156f60260b3af42" Jul 13 17:07:05.907788 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:07:05.907768 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72e8ecfcb766c4a1881dc51c2909f45cbf1d5bf35cca9fe01156f60260b3af42\": container with ID starting with 72e8ecfcb766c4a1881dc51c2909f45cbf1d5bf35cca9fe01156f60260b3af42 not found: ID does not exist" containerID="72e8ecfcb766c4a1881dc51c2909f45cbf1d5bf35cca9fe01156f60260b3af42" Jul 13 17:07:05.907841 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:07:05.907796 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72e8ecfcb766c4a1881dc51c2909f45cbf1d5bf35cca9fe01156f60260b3af42"} err="failed to get container status \"72e8ecfcb766c4a1881dc51c2909f45cbf1d5bf35cca9fe01156f60260b3af42\": rpc error: code = NotFound desc = could not find container \"72e8ecfcb766c4a1881dc51c2909f45cbf1d5bf35cca9fe01156f60260b3af42\": container with ID starting with 72e8ecfcb766c4a1881dc51c2909f45cbf1d5bf35cca9fe01156f60260b3af42 not found: ID does not exist" Jul 13 17:07:05.918748 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:07:05.918724 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5878cf7589-sgggs"] Jul 13 17:07:05.923849 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:07:05.923832 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5878cf7589-sgggs"] Jul 13 17:07:06.138522 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:07:06.138494 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46bae189-2ca3-416c-bcb7-42b1283f71b8" path="/var/lib/kubelet/pods/46bae189-2ca3-416c-bcb7-42b1283f71b8/volumes" Jul 13 17:07:07.957313 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:07:07.957268 2569 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc7d8764f-h64lg" podUID="e3896c87-e2a0-4309-aa5d-bcf65d9e67e9" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 13 17:07:08.913261 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:07:08.913227 2569 generic.go:356] "Generic (PLEG): container finished" podID="7ffdcb20-b7c8-468e-9c1f-c68e9e588a93" containerID="23c05a194122135fa1770163e0b03ae06e86935515b0e7781c1519b017bb0788" exitCode=0 Jul 13 17:07:08.913406 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:07:08.913303 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-j2xl7" event={"ID":"7ffdcb20-b7c8-468e-9c1f-c68e9e588a93","Type":"ContainerDied","Data":"23c05a194122135fa1770163e0b03ae06e86935515b0e7781c1519b017bb0788"} Jul 13 17:07:08.913696 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:07:08.913676 2569 scope.go:117] "RemoveContainer" containerID="23c05a194122135fa1770163e0b03ae06e86935515b0e7781c1519b017bb0788" Jul 13 17:07:09.918930 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:07:09.918894 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-j2xl7" event={"ID":"7ffdcb20-b7c8-468e-9c1f-c68e9e588a93","Type":"ContainerStarted","Data":"9f80b72eabe461e922f377819c99bf6ddb2849fea8dac9c541f9fb89e892c426"} Jul 13 17:07:13.936059 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:07:13.936026 2569 generic.go:356] "Generic (PLEG): container finished" podID="ddbfed7e-ef66-42cf-a6e3-481f1da70624" containerID="21b53a10c667b1a9efb99a3b032ebee4c6837a9ff3101be17143f6458bd4712e" exitCode=0 Jul 13 17:07:13.936429 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:07:13.936094 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-fskzj" event={"ID":"ddbfed7e-ef66-42cf-a6e3-481f1da70624","Type":"ContainerDied","Data":"21b53a10c667b1a9efb99a3b032ebee4c6837a9ff3101be17143f6458bd4712e"} Jul 13 17:07:13.936497 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:07:13.936430 2569 scope.go:117] "RemoveContainer" containerID="21b53a10c667b1a9efb99a3b032ebee4c6837a9ff3101be17143f6458bd4712e" Jul 13 17:07:14.942029 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:07:14.941990 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-fskzj" event={"ID":"ddbfed7e-ef66-42cf-a6e3-481f1da70624","Type":"ContainerStarted","Data":"bbd2f90fec2488153b0a04d15028a1e3f143a078850549cb43e88637766ca5d2"} Jul 13 17:07:14.943372 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:07:14.943351 2569 generic.go:356] "Generic (PLEG): container finished" podID="bd4ce4fa-6339-4a0c-8260-c6a9538543c2" containerID="fe15154c254dfa806e3a2628d929864811cf5705e9f217c03e41a00ea808444c" exitCode=0 Jul 13 17:07:14.943491 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:07:14.943407 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-7s9dh" event={"ID":"bd4ce4fa-6339-4a0c-8260-c6a9538543c2","Type":"ContainerDied","Data":"fe15154c254dfa806e3a2628d929864811cf5705e9f217c03e41a00ea808444c"} Jul 13 17:07:14.943653 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:07:14.943640 2569 scope.go:117] "RemoveContainer" containerID="fe15154c254dfa806e3a2628d929864811cf5705e9f217c03e41a00ea808444c" Jul 13 17:07:15.951135 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:07:15.951086 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-7s9dh" event={"ID":"bd4ce4fa-6339-4a0c-8260-c6a9538543c2","Type":"ContainerStarted","Data":"e06c137eadd4dc42f440e4de02ded860ca1726ada77b716d9f131e6c97625f7f"} Jul 13 17:07:17.957351 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:07:17.957314 2569 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc7d8764f-h64lg" podUID="e3896c87-e2a0-4309-aa5d-bcf65d9e67e9" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 13 17:07:27.957056 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:07:27.957009 2569 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc7d8764f-h64lg" podUID="e3896c87-e2a0-4309-aa5d-bcf65d9e67e9" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 13 17:07:27.957400 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:07:27.957096 2569 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc7d8764f-h64lg" Jul 13 17:07:27.957664 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:07:27.957643 2569 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"e302c507a097aaf597af06467f894f4946d6cdda4a218aae9af2e47bb8548299"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc7d8764f-h64lg" containerMessage="Container service-proxy failed liveness probe, will be restarted" Jul 13 17:07:27.957711 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:07:27.957688 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc7d8764f-h64lg" podUID="e3896c87-e2a0-4309-aa5d-bcf65d9e67e9" containerName="service-proxy" containerID="cri-o://e302c507a097aaf597af06467f894f4946d6cdda4a218aae9af2e47bb8548299" gracePeriod=30 Jul 13 17:07:29.003525 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:07:29.003492 2569 generic.go:356] "Generic (PLEG): container finished" podID="e3896c87-e2a0-4309-aa5d-bcf65d9e67e9" containerID="e302c507a097aaf597af06467f894f4946d6cdda4a218aae9af2e47bb8548299" exitCode=2 Jul 13 17:07:29.003897 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:07:29.003535 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc7d8764f-h64lg" event={"ID":"e3896c87-e2a0-4309-aa5d-bcf65d9e67e9","Type":"ContainerDied","Data":"e302c507a097aaf597af06467f894f4946d6cdda4a218aae9af2e47bb8548299"} Jul 13 17:07:29.003897 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:07:29.003583 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fc7d8764f-h64lg" event={"ID":"e3896c87-e2a0-4309-aa5d-bcf65d9e67e9","Type":"ContainerStarted","Data":"be3ea509987e5c67879e007a699953036d55aa02e613dac7762e1cb00a15ebd8"} Jul 13 17:10:04.029228 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:10:04.029202 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-hzcql_c40e6e88-ab3f-4e3c-a0e7-2400bda333eb/console-operator/1.log" Jul 13 17:10:04.036397 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:10:04.036372 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-hzcql_c40e6e88-ab3f-4e3c-a0e7-2400bda333eb/console-operator/1.log" Jul 13 17:10:04.037667 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:10:04.037647 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q2pnb_6a531b70-4a58-4752-8fa8-5ba634b542ea/ovn-acl-logging/0.log" Jul 13 17:10:04.042373 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:10:04.042352 2569 kubelet.go:1648] "Image garbage collection succeeded" Jul 13 17:10:04.046077 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:10:04.046057 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q2pnb_6a531b70-4a58-4752-8fa8-5ba634b542ea/ovn-acl-logging/0.log" Jul 13 17:11:31.972951 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:31.972874 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-dfb6c5f74-q452h"] Jul 13 17:11:31.973376 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:31.973176 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46bae189-2ca3-416c-bcb7-42b1283f71b8" containerName="registry" Jul 13 17:11:31.973376 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:31.973192 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="46bae189-2ca3-416c-bcb7-42b1283f71b8" containerName="registry" Jul 13 17:11:31.973376 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:31.973262 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="46bae189-2ca3-416c-bcb7-42b1283f71b8" containerName="registry" Jul 13 17:11:31.975383 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:31.975366 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-dfb6c5f74-q452h" Jul 13 17:11:31.977419 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:31.977396 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 13 17:11:31.977537 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:31.977463 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 13 17:11:31.977537 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:31.977463 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-kx26v\"" Jul 13 17:11:31.977671 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:31.977623 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 13 17:11:31.977777 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:31.977754 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 13 17:11:31.991751 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:31.991731 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-dfb6c5f74-q452h"] Jul 13 17:11:32.070753 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:32.070726 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d5df4bee-43d7-4d5d-bc22-70b20478df18-tmp\") pod \"opendatahub-operator-controller-manager-dfb6c5f74-q452h\" (UID: \"d5df4bee-43d7-4d5d-bc22-70b20478df18\") " pod="opendatahub/opendatahub-operator-controller-manager-dfb6c5f74-q452h" Jul 13 17:11:32.070861 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:32.070777 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d5df4bee-43d7-4d5d-bc22-70b20478df18-apiservice-cert\") pod \"opendatahub-operator-controller-manager-dfb6c5f74-q452h\" (UID: \"d5df4bee-43d7-4d5d-bc22-70b20478df18\") " pod="opendatahub/opendatahub-operator-controller-manager-dfb6c5f74-q452h" Jul 13 17:11:32.070861 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:32.070806 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d5df4bee-43d7-4d5d-bc22-70b20478df18-webhook-cert\") pod \"opendatahub-operator-controller-manager-dfb6c5f74-q452h\" (UID: \"d5df4bee-43d7-4d5d-bc22-70b20478df18\") " pod="opendatahub/opendatahub-operator-controller-manager-dfb6c5f74-q452h" Jul 13 17:11:32.070861 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:32.070856 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzt8w\" (UniqueName: \"kubernetes.io/projected/d5df4bee-43d7-4d5d-bc22-70b20478df18-kube-api-access-hzt8w\") pod \"opendatahub-operator-controller-manager-dfb6c5f74-q452h\" (UID: \"d5df4bee-43d7-4d5d-bc22-70b20478df18\") " pod="opendatahub/opendatahub-operator-controller-manager-dfb6c5f74-q452h" Jul 13 17:11:32.070995 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:32.070902 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d5df4bee-43d7-4d5d-bc22-70b20478df18-manifests\") pod \"opendatahub-operator-controller-manager-dfb6c5f74-q452h\" (UID: \"d5df4bee-43d7-4d5d-bc22-70b20478df18\") " pod="opendatahub/opendatahub-operator-controller-manager-dfb6c5f74-q452h" Jul 13 17:11:32.172244 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:32.172205 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d5df4bee-43d7-4d5d-bc22-70b20478df18-apiservice-cert\") pod \"opendatahub-operator-controller-manager-dfb6c5f74-q452h\" (UID: \"d5df4bee-43d7-4d5d-bc22-70b20478df18\") " pod="opendatahub/opendatahub-operator-controller-manager-dfb6c5f74-q452h" Jul 13 17:11:32.172244 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:32.172258 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d5df4bee-43d7-4d5d-bc22-70b20478df18-webhook-cert\") pod \"opendatahub-operator-controller-manager-dfb6c5f74-q452h\" (UID: \"d5df4bee-43d7-4d5d-bc22-70b20478df18\") " pod="opendatahub/opendatahub-operator-controller-manager-dfb6c5f74-q452h" Jul 13 17:11:32.172474 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:32.172311 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzt8w\" (UniqueName: \"kubernetes.io/projected/d5df4bee-43d7-4d5d-bc22-70b20478df18-kube-api-access-hzt8w\") pod \"opendatahub-operator-controller-manager-dfb6c5f74-q452h\" (UID: \"d5df4bee-43d7-4d5d-bc22-70b20478df18\") " pod="opendatahub/opendatahub-operator-controller-manager-dfb6c5f74-q452h" Jul 13 17:11:32.172474 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:32.172341 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d5df4bee-43d7-4d5d-bc22-70b20478df18-manifests\") pod \"opendatahub-operator-controller-manager-dfb6c5f74-q452h\" (UID: \"d5df4bee-43d7-4d5d-bc22-70b20478df18\") " pod="opendatahub/opendatahub-operator-controller-manager-dfb6c5f74-q452h" Jul 13 17:11:32.172474 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:32.172382 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d5df4bee-43d7-4d5d-bc22-70b20478df18-tmp\") pod \"opendatahub-operator-controller-manager-dfb6c5f74-q452h\" (UID: \"d5df4bee-43d7-4d5d-bc22-70b20478df18\") " pod="opendatahub/opendatahub-operator-controller-manager-dfb6c5f74-q452h" Jul 13 17:11:32.172790 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:32.172751 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d5df4bee-43d7-4d5d-bc22-70b20478df18-manifests\") pod \"opendatahub-operator-controller-manager-dfb6c5f74-q452h\" (UID: \"d5df4bee-43d7-4d5d-bc22-70b20478df18\") " pod="opendatahub/opendatahub-operator-controller-manager-dfb6c5f74-q452h" Jul 13 17:11:32.174512 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:32.174492 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d5df4bee-43d7-4d5d-bc22-70b20478df18-tmp\") pod \"opendatahub-operator-controller-manager-dfb6c5f74-q452h\" (UID: \"d5df4bee-43d7-4d5d-bc22-70b20478df18\") " pod="opendatahub/opendatahub-operator-controller-manager-dfb6c5f74-q452h" Jul 13 17:11:32.174751 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:32.174733 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d5df4bee-43d7-4d5d-bc22-70b20478df18-webhook-cert\") pod \"opendatahub-operator-controller-manager-dfb6c5f74-q452h\" (UID: \"d5df4bee-43d7-4d5d-bc22-70b20478df18\") " pod="opendatahub/opendatahub-operator-controller-manager-dfb6c5f74-q452h" Jul 13 17:11:32.174823 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:32.174780 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d5df4bee-43d7-4d5d-bc22-70b20478df18-apiservice-cert\") pod \"opendatahub-operator-controller-manager-dfb6c5f74-q452h\" (UID: \"d5df4bee-43d7-4d5d-bc22-70b20478df18\") " pod="opendatahub/opendatahub-operator-controller-manager-dfb6c5f74-q452h" Jul 13 17:11:32.179976 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:32.179947 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzt8w\" (UniqueName: \"kubernetes.io/projected/d5df4bee-43d7-4d5d-bc22-70b20478df18-kube-api-access-hzt8w\") pod \"opendatahub-operator-controller-manager-dfb6c5f74-q452h\" (UID: \"d5df4bee-43d7-4d5d-bc22-70b20478df18\") " pod="opendatahub/opendatahub-operator-controller-manager-dfb6c5f74-q452h" Jul 13 17:11:32.285894 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:32.285861 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-dfb6c5f74-q452h" Jul 13 17:11:32.428095 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:32.428070 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-dfb6c5f74-q452h"] Jul 13 17:11:32.430828 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:11:32.430803 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5df4bee_43d7_4d5d_bc22_70b20478df18.slice/crio-2812ee4bd904454ded75ad9581c407622960c45da4ed430201940b7cfdac6baf WatchSource:0}: Error finding container 2812ee4bd904454ded75ad9581c407622960c45da4ed430201940b7cfdac6baf: Status 404 returned error can't find the container with id 2812ee4bd904454ded75ad9581c407622960c45da4ed430201940b7cfdac6baf Jul 13 17:11:32.432598 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:32.432579 2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 17:11:32.904819 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:32.904785 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dfb6c5f74-q452h" event={"ID":"d5df4bee-43d7-4d5d-bc22-70b20478df18","Type":"ContainerStarted","Data":"2812ee4bd904454ded75ad9581c407622960c45da4ed430201940b7cfdac6baf"} Jul 13 17:11:35.919542 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:35.919511 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dfb6c5f74-q452h" event={"ID":"d5df4bee-43d7-4d5d-bc22-70b20478df18","Type":"ContainerStarted","Data":"aad7fd4ba47cf54cc64d1e17181f4fedadd490c6624106e05bad70843bcc954b"} Jul 13 17:11:37.929870 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:37.929831 2569 generic.go:356] "Generic (PLEG): container finished" podID="d5df4bee-43d7-4d5d-bc22-70b20478df18" containerID="aad7fd4ba47cf54cc64d1e17181f4fedadd490c6624106e05bad70843bcc954b" exitCode=0 Jul 13 17:11:37.930251 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:37.929905 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dfb6c5f74-q452h" event={"ID":"d5df4bee-43d7-4d5d-bc22-70b20478df18","Type":"ContainerDied","Data":"aad7fd4ba47cf54cc64d1e17181f4fedadd490c6624106e05bad70843bcc954b"} Jul 13 17:11:38.935495 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:38.935461 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dfb6c5f74-q452h" event={"ID":"d5df4bee-43d7-4d5d-bc22-70b20478df18","Type":"ContainerStarted","Data":"14579cdd876db2e96bc0909fd2f35418ff295b339b2237cebd8640f47e10fb82"} Jul 13 17:11:38.935902 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:38.935698 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-dfb6c5f74-q452h" Jul 13 17:11:38.956812 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:38.956762 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-dfb6c5f74-q452h" podStartSLOduration=3.121894024 podStartE2EDuration="7.95674724s" totalImagesPullingTime="3.751611523s" totalInitContainerRuntime="1.083241693s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:11:31 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:11:38.955401259 +0000 UTC m=+395.353059648" watchObservedRunningTime="2026-07-13 17:11:38.95674724 +0000 UTC m=+395.354405619" Jul 13 17:11:49.942972 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:49.942940 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-dfb6c5f74-q452h" Jul 13 17:11:52.305931 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:52.305893 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-f6649c567-rhl5t"] Jul 13 17:11:52.316180 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:52.316158 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-f6649c567-rhl5t" Jul 13 17:11:52.320465 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:52.320438 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-fth2m\"" Jul 13 17:11:52.320606 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:52.320461 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 13 17:11:52.320606 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:52.320484 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 13 17:11:52.320606 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:52.320497 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 13 17:11:52.320606 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:52.320466 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 13 17:11:52.320606 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:52.320505 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 13 17:11:52.323058 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:52.323036 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-f6649c567-rhl5t"] Jul 13 17:11:52.426012 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:52.425984 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4aa024cf-46c7-429a-b180-ef267e7b69b5-manager-config\") pod \"lws-controller-manager-f6649c567-rhl5t\" (UID: \"4aa024cf-46c7-429a-b180-ef267e7b69b5\") " pod="openshift-lws-operator/lws-controller-manager-f6649c567-rhl5t" Jul 13 17:11:52.426119 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:52.426028 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4aa024cf-46c7-429a-b180-ef267e7b69b5-metrics-cert\") pod \"lws-controller-manager-f6649c567-rhl5t\" (UID: \"4aa024cf-46c7-429a-b180-ef267e7b69b5\") " pod="openshift-lws-operator/lws-controller-manager-f6649c567-rhl5t" Jul 13 17:11:52.426119 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:52.426055 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kcvg\" (UniqueName: \"kubernetes.io/projected/4aa024cf-46c7-429a-b180-ef267e7b69b5-kube-api-access-7kcvg\") pod \"lws-controller-manager-f6649c567-rhl5t\" (UID: \"4aa024cf-46c7-429a-b180-ef267e7b69b5\") " pod="openshift-lws-operator/lws-controller-manager-f6649c567-rhl5t" Jul 13 17:11:52.426119 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:52.426099 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4aa024cf-46c7-429a-b180-ef267e7b69b5-cert\") pod \"lws-controller-manager-f6649c567-rhl5t\" (UID: \"4aa024cf-46c7-429a-b180-ef267e7b69b5\") " pod="openshift-lws-operator/lws-controller-manager-f6649c567-rhl5t" Jul 13 17:11:52.526686 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:52.526654 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4aa024cf-46c7-429a-b180-ef267e7b69b5-manager-config\") pod \"lws-controller-manager-f6649c567-rhl5t\" (UID: \"4aa024cf-46c7-429a-b180-ef267e7b69b5\") " pod="openshift-lws-operator/lws-controller-manager-f6649c567-rhl5t" Jul 13 17:11:52.526827 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:52.526703 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4aa024cf-46c7-429a-b180-ef267e7b69b5-metrics-cert\") pod \"lws-controller-manager-f6649c567-rhl5t\" (UID: \"4aa024cf-46c7-429a-b180-ef267e7b69b5\") " pod="openshift-lws-operator/lws-controller-manager-f6649c567-rhl5t" Jul 13 17:11:52.526827 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:52.526731 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kcvg\" (UniqueName: \"kubernetes.io/projected/4aa024cf-46c7-429a-b180-ef267e7b69b5-kube-api-access-7kcvg\") pod \"lws-controller-manager-f6649c567-rhl5t\" (UID: \"4aa024cf-46c7-429a-b180-ef267e7b69b5\") " pod="openshift-lws-operator/lws-controller-manager-f6649c567-rhl5t" Jul 13 17:11:52.526827 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:52.526751 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4aa024cf-46c7-429a-b180-ef267e7b69b5-cert\") pod \"lws-controller-manager-f6649c567-rhl5t\" (UID: \"4aa024cf-46c7-429a-b180-ef267e7b69b5\") " pod="openshift-lws-operator/lws-controller-manager-f6649c567-rhl5t" Jul 13 17:11:52.527428 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:52.527403 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4aa024cf-46c7-429a-b180-ef267e7b69b5-manager-config\") pod \"lws-controller-manager-f6649c567-rhl5t\" (UID: \"4aa024cf-46c7-429a-b180-ef267e7b69b5\") " pod="openshift-lws-operator/lws-controller-manager-f6649c567-rhl5t" Jul 13 17:11:52.529253 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:52.529231 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4aa024cf-46c7-429a-b180-ef267e7b69b5-cert\") pod \"lws-controller-manager-f6649c567-rhl5t\" (UID: \"4aa024cf-46c7-429a-b180-ef267e7b69b5\") " pod="openshift-lws-operator/lws-controller-manager-f6649c567-rhl5t" Jul 13 17:11:52.529357 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:52.529259 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4aa024cf-46c7-429a-b180-ef267e7b69b5-metrics-cert\") pod \"lws-controller-manager-f6649c567-rhl5t\" (UID: \"4aa024cf-46c7-429a-b180-ef267e7b69b5\") " pod="openshift-lws-operator/lws-controller-manager-f6649c567-rhl5t" Jul 13 17:11:52.533585 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:52.533540 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kcvg\" (UniqueName: \"kubernetes.io/projected/4aa024cf-46c7-429a-b180-ef267e7b69b5-kube-api-access-7kcvg\") pod \"lws-controller-manager-f6649c567-rhl5t\" (UID: \"4aa024cf-46c7-429a-b180-ef267e7b69b5\") " pod="openshift-lws-operator/lws-controller-manager-f6649c567-rhl5t" Jul 13 17:11:52.627624 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:52.627536 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-f6649c567-rhl5t" Jul 13 17:11:52.763506 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:52.763476 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-f6649c567-rhl5t"] Jul 13 17:11:52.766558 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:11:52.766531 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4aa024cf_46c7_429a_b180_ef267e7b69b5.slice/crio-abfffa395064da9db6e3bb3fcd9260aafb5d5810859ae580c90d2f8d971fa8a9 WatchSource:0}: Error finding container abfffa395064da9db6e3bb3fcd9260aafb5d5810859ae580c90d2f8d971fa8a9: Status 404 returned error can't find the container with id abfffa395064da9db6e3bb3fcd9260aafb5d5810859ae580c90d2f8d971fa8a9 Jul 13 17:11:53.001055 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:53.000973 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-f6649c567-rhl5t" event={"ID":"4aa024cf-46c7-429a-b180-ef267e7b69b5","Type":"ContainerStarted","Data":"abfffa395064da9db6e3bb3fcd9260aafb5d5810859ae580c90d2f8d971fa8a9"} Jul 13 17:11:56.016583 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:56.016533 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-f6649c567-rhl5t" event={"ID":"4aa024cf-46c7-429a-b180-ef267e7b69b5","Type":"ContainerStarted","Data":"711e126280980fa2f4a2f622bd3561d59ccb343a8d58576491f5459c4e1692df"} Jul 13 17:11:56.016971 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:56.016621 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-f6649c567-rhl5t" Jul 13 17:11:56.044503 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:11:56.044457 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-f6649c567-rhl5t" podStartSLOduration=1.595373571 podStartE2EDuration="4.044443497s" totalImagesPullingTime="2.449069926s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:11:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:11:56.043529319 +0000 UTC m=+412.441187709" watchObservedRunningTime="2026-07-13 17:11:56.044443497 +0000 UTC m=+412.442101876" Jul 13 17:12:07.022951 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:12:07.022921 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-f6649c567-rhl5t" Jul 13 17:12:53.272537 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:12:53.272502 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tx5zp"] Jul 13 17:12:53.276538 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:12:53.276518 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-tx5zp" Jul 13 17:12:53.278755 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:12:53.278733 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 13 17:12:53.278755 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:12:53.278748 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 13 17:12:53.279351 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:12:53.279226 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-9wqk2\"" Jul 13 17:12:53.287165 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:12:53.287140 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tx5zp"] Jul 13 17:12:53.367517 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:12:53.367486 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmzgz\" (UniqueName: \"kubernetes.io/projected/4ab907ba-ea50-4b3e-9383-fde038aa91bf-kube-api-access-mmzgz\") pod \"kuadrant-operator-catalog-tx5zp\" (UID: \"4ab907ba-ea50-4b3e-9383-fde038aa91bf\") " pod="kuadrant-system/kuadrant-operator-catalog-tx5zp" Jul 13 17:12:53.468451 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:12:53.468422 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmzgz\" (UniqueName: \"kubernetes.io/projected/4ab907ba-ea50-4b3e-9383-fde038aa91bf-kube-api-access-mmzgz\") pod \"kuadrant-operator-catalog-tx5zp\" (UID: \"4ab907ba-ea50-4b3e-9383-fde038aa91bf\") " pod="kuadrant-system/kuadrant-operator-catalog-tx5zp" Jul 13 17:12:53.476426 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:12:53.476404 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmzgz\" (UniqueName: \"kubernetes.io/projected/4ab907ba-ea50-4b3e-9383-fde038aa91bf-kube-api-access-mmzgz\") pod \"kuadrant-operator-catalog-tx5zp\" (UID: \"4ab907ba-ea50-4b3e-9383-fde038aa91bf\") " pod="kuadrant-system/kuadrant-operator-catalog-tx5zp" Jul 13 17:12:53.587356 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:12:53.587327 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-tx5zp" Jul 13 17:12:53.610159 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:12:53.610136 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tx5zp"] Jul 13 17:12:53.714380 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:12:53.714352 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tx5zp"] Jul 13 17:12:53.716968 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:12:53.716939 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ab907ba_ea50_4b3e_9383_fde038aa91bf.slice/crio-dad8632cda344a7093edb457dbceead77d2885ffe8445e27704a9aa17a0ca3ee WatchSource:0}: Error finding container dad8632cda344a7093edb457dbceead77d2885ffe8445e27704a9aa17a0ca3ee: Status 404 returned error can't find the container with id dad8632cda344a7093edb457dbceead77d2885ffe8445e27704a9aa17a0ca3ee Jul 13 17:12:53.821340 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:12:53.821313 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-d9njc"] Jul 13 17:12:53.824643 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:12:53.824626 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-d9njc" Jul 13 17:12:53.831360 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:12:53.831305 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-d9njc"] Jul 13 17:12:53.870728 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:12:53.870666 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58qn7\" (UniqueName: \"kubernetes.io/projected/0b9da11a-0dfd-42e1-891e-2891173db7fa-kube-api-access-58qn7\") pod \"kuadrant-operator-catalog-d9njc\" (UID: \"0b9da11a-0dfd-42e1-891e-2891173db7fa\") " pod="kuadrant-system/kuadrant-operator-catalog-d9njc" Jul 13 17:12:53.971162 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:12:53.971134 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58qn7\" (UniqueName: \"kubernetes.io/projected/0b9da11a-0dfd-42e1-891e-2891173db7fa-kube-api-access-58qn7\") pod \"kuadrant-operator-catalog-d9njc\" (UID: \"0b9da11a-0dfd-42e1-891e-2891173db7fa\") " pod="kuadrant-system/kuadrant-operator-catalog-d9njc" Jul 13 17:12:53.979403 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:12:53.979377 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-58qn7\" (UniqueName: \"kubernetes.io/projected/0b9da11a-0dfd-42e1-891e-2891173db7fa-kube-api-access-58qn7\") pod \"kuadrant-operator-catalog-d9njc\" (UID: \"0b9da11a-0dfd-42e1-891e-2891173db7fa\") " pod="kuadrant-system/kuadrant-operator-catalog-d9njc" Jul 13 17:12:54.133783 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:12:54.133697 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-d9njc" Jul 13 17:12:54.261293 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:12:54.261256 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-tx5zp" event={"ID":"4ab907ba-ea50-4b3e-9383-fde038aa91bf","Type":"ContainerStarted","Data":"dad8632cda344a7093edb457dbceead77d2885ffe8445e27704a9aa17a0ca3ee"} Jul 13 17:12:54.281019 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:12:54.280952 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-d9njc"] Jul 13 17:12:54.303089 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:12:54.303059 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b9da11a_0dfd_42e1_891e_2891173db7fa.slice/crio-390613fc977287301649916dc9765219b812bd77746fb876e6337c390c0db513 WatchSource:0}: Error finding container 390613fc977287301649916dc9765219b812bd77746fb876e6337c390c0db513: Status 404 returned error can't find the container with id 390613fc977287301649916dc9765219b812bd77746fb876e6337c390c0db513 Jul 13 17:12:55.268141 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:12:55.268102 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-d9njc" event={"ID":"0b9da11a-0dfd-42e1-891e-2891173db7fa","Type":"ContainerStarted","Data":"390613fc977287301649916dc9765219b812bd77746fb876e6337c390c0db513"} Jul 13 17:12:56.273527 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:12:56.273422 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-d9njc" event={"ID":"0b9da11a-0dfd-42e1-891e-2891173db7fa","Type":"ContainerStarted","Data":"2eaf7a494100a1559859a1d1e03a7ab77e075d1c86c96fc1b5ce74cbcc67ce17"} Jul 13 17:12:56.274851 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:12:56.274827 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-tx5zp" event={"ID":"4ab907ba-ea50-4b3e-9383-fde038aa91bf","Type":"ContainerStarted","Data":"101d8dcb176cd8b9ddf3528a009349dd40656ae11f86cb2a13df8799f52342e3"} Jul 13 17:12:56.274939 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:12:56.274912 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-tx5zp" podUID="4ab907ba-ea50-4b3e-9383-fde038aa91bf" containerName="registry-server" containerID="cri-o://101d8dcb176cd8b9ddf3528a009349dd40656ae11f86cb2a13df8799f52342e3" gracePeriod=2 Jul 13 17:12:56.290593 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:12:56.290532 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-d9njc" podStartSLOduration=1.788619282 podStartE2EDuration="3.290518765s" totalImagesPullingTime="1.501899483s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:12:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:12:56.288661301 +0000 UTC m=+472.686319679" watchObservedRunningTime="2026-07-13 17:12:56.290518765 +0000 UTC m=+472.688177147" Jul 13 17:12:56.304712 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:12:56.304678 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-tx5zp" podStartSLOduration=1.219844346 podStartE2EDuration="3.304666152s" totalImagesPullingTime="2.084821806s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:12:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:12:56.303759118 +0000 UTC m=+472.701417523" watchObservedRunningTime="2026-07-13 17:12:56.304666152 +0000 UTC m=+472.702324531" Jul 13 17:12:56.526499 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:12:56.526442 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-tx5zp" Jul 13 17:12:56.588611 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:12:56.588578 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mmzgz\" (UniqueName: \"kubernetes.io/projected/4ab907ba-ea50-4b3e-9383-fde038aa91bf-kube-api-access-mmzgz\") pod \"4ab907ba-ea50-4b3e-9383-fde038aa91bf\" (UID: \"4ab907ba-ea50-4b3e-9383-fde038aa91bf\") " Jul 13 17:12:56.590614 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:12:56.590593 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ab907ba-ea50-4b3e-9383-fde038aa91bf-kube-api-access-mmzgz" (OuterVolumeSpecName: "kube-api-access-mmzgz") pod "4ab907ba-ea50-4b3e-9383-fde038aa91bf" (UID: "4ab907ba-ea50-4b3e-9383-fde038aa91bf"). InnerVolumeSpecName "kube-api-access-mmzgz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:12:56.689078 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:12:56.689051 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mmzgz\" (UniqueName: \"kubernetes.io/projected/4ab907ba-ea50-4b3e-9383-fde038aa91bf-kube-api-access-mmzgz\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:12:57.280186 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:12:57.280107 2569 generic.go:356] "Generic (PLEG): container finished" podID="4ab907ba-ea50-4b3e-9383-fde038aa91bf" containerID="101d8dcb176cd8b9ddf3528a009349dd40656ae11f86cb2a13df8799f52342e3" exitCode=0 Jul 13 17:12:57.280186 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:12:57.280148 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-tx5zp" Jul 13 17:12:57.280636 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:12:57.280193 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-tx5zp" event={"ID":"4ab907ba-ea50-4b3e-9383-fde038aa91bf","Type":"ContainerDied","Data":"101d8dcb176cd8b9ddf3528a009349dd40656ae11f86cb2a13df8799f52342e3"} Jul 13 17:12:57.280636 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:12:57.280243 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-tx5zp" event={"ID":"4ab907ba-ea50-4b3e-9383-fde038aa91bf","Type":"ContainerDied","Data":"dad8632cda344a7093edb457dbceead77d2885ffe8445e27704a9aa17a0ca3ee"} Jul 13 17:12:57.280636 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:12:57.280271 2569 scope.go:117] "RemoveContainer" containerID="101d8dcb176cd8b9ddf3528a009349dd40656ae11f86cb2a13df8799f52342e3" Jul 13 17:12:57.311919 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:12:57.288935 2569 scope.go:117] "RemoveContainer" containerID="101d8dcb176cd8b9ddf3528a009349dd40656ae11f86cb2a13df8799f52342e3" Jul 13 17:12:57.311919 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:12:57.289258 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"101d8dcb176cd8b9ddf3528a009349dd40656ae11f86cb2a13df8799f52342e3\": container with ID starting with 101d8dcb176cd8b9ddf3528a009349dd40656ae11f86cb2a13df8799f52342e3 not found: ID does not exist" containerID="101d8dcb176cd8b9ddf3528a009349dd40656ae11f86cb2a13df8799f52342e3" Jul 13 17:12:57.311919 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:12:57.289294 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"101d8dcb176cd8b9ddf3528a009349dd40656ae11f86cb2a13df8799f52342e3"} err="failed to get container status \"101d8dcb176cd8b9ddf3528a009349dd40656ae11f86cb2a13df8799f52342e3\": rpc error: code = NotFound desc = could not find container \"101d8dcb176cd8b9ddf3528a009349dd40656ae11f86cb2a13df8799f52342e3\": container with ID starting with 101d8dcb176cd8b9ddf3528a009349dd40656ae11f86cb2a13df8799f52342e3 not found: ID does not exist" Jul 13 17:12:57.311919 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:12:57.302500 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tx5zp"] Jul 13 17:12:57.311919 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:12:57.305973 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tx5zp"] Jul 13 17:12:58.137074 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:12:58.137044 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ab907ba-ea50-4b3e-9383-fde038aa91bf" path="/var/lib/kubelet/pods/4ab907ba-ea50-4b3e-9383-fde038aa91bf/volumes" Jul 13 17:13:04.137508 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:13:04.137480 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-d9njc" Jul 13 17:13:04.137867 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:13:04.137529 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-d9njc" Jul 13 17:13:04.157080 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:13:04.157057 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-d9njc" Jul 13 17:13:04.331008 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:13:04.330938 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-d9njc" Jul 13 17:13:27.240503 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:13:27.240464 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-vhrj6"] Jul 13 17:13:27.241000 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:13:27.240797 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ab907ba-ea50-4b3e-9383-fde038aa91bf" containerName="registry-server" Jul 13 17:13:27.241000 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:13:27.240809 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ab907ba-ea50-4b3e-9383-fde038aa91bf" containerName="registry-server" Jul 13 17:13:27.241000 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:13:27.240873 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ab907ba-ea50-4b3e-9383-fde038aa91bf" containerName="registry-server" Jul 13 17:13:27.248214 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:13:27.248193 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-vhrj6" Jul 13 17:13:27.250235 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:13:27.250214 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-hjcd8\"" Jul 13 17:13:27.250531 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:13:27.250513 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 13 17:13:27.262461 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:13:27.262438 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-vhrj6"] Jul 13 17:13:27.313752 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:13:27.313729 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2drp\" (UniqueName: \"kubernetes.io/projected/5784681e-3fbe-4ffe-b796-9cd3589847d2-kube-api-access-w2drp\") pod \"dns-operator-controller-manager-864dd78c4c-vhrj6\" (UID: \"5784681e-3fbe-4ffe-b796-9cd3589847d2\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-vhrj6" Jul 13 17:13:27.414482 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:13:27.414456 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2drp\" (UniqueName: \"kubernetes.io/projected/5784681e-3fbe-4ffe-b796-9cd3589847d2-kube-api-access-w2drp\") pod \"dns-operator-controller-manager-864dd78c4c-vhrj6\" (UID: \"5784681e-3fbe-4ffe-b796-9cd3589847d2\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-vhrj6" Jul 13 17:13:27.422931 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:13:27.422903 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2drp\" (UniqueName: \"kubernetes.io/projected/5784681e-3fbe-4ffe-b796-9cd3589847d2-kube-api-access-w2drp\") pod \"dns-operator-controller-manager-864dd78c4c-vhrj6\" (UID: \"5784681e-3fbe-4ffe-b796-9cd3589847d2\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-vhrj6" Jul 13 17:13:27.558058 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:13:27.558038 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-vhrj6" Jul 13 17:13:27.691022 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:13:27.690876 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-vhrj6"] Jul 13 17:13:27.693893 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:13:27.693866 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5784681e_3fbe_4ffe_b796_9cd3589847d2.slice/crio-40ef24941a2486331f1588e2a85fbc78b55e43d98740e0bb10bd85e0adcd4d8f WatchSource:0}: Error finding container 40ef24941a2486331f1588e2a85fbc78b55e43d98740e0bb10bd85e0adcd4d8f: Status 404 returned error can't find the container with id 40ef24941a2486331f1588e2a85fbc78b55e43d98740e0bb10bd85e0adcd4d8f Jul 13 17:13:28.407017 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:13:28.406983 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-vhrj6" event={"ID":"5784681e-3fbe-4ffe-b796-9cd3589847d2","Type":"ContainerStarted","Data":"40ef24941a2486331f1588e2a85fbc78b55e43d98740e0bb10bd85e0adcd4d8f"} Jul 13 17:13:30.416484 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:13:30.416449 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-vhrj6" event={"ID":"5784681e-3fbe-4ffe-b796-9cd3589847d2","Type":"ContainerStarted","Data":"8cd9d7c35e6f8602f54a668c7bcf55765ef0737998aa6ff858c3c1ee0000cea3"} Jul 13 17:13:30.416866 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:13:30.416518 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-vhrj6" Jul 13 17:13:30.434065 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:13:30.434011 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-vhrj6" podStartSLOduration=1.109928131 podStartE2EDuration="3.433996891s" totalImagesPullingTime="2.32406876s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:13:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:13:30.43258105 +0000 UTC m=+506.830239423" watchObservedRunningTime="2026-07-13 17:13:30.433996891 +0000 UTC m=+506.831655269" Jul 13 17:13:31.652255 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:13:31.652222 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xm87m"] Jul 13 17:13:31.655434 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:13:31.655418 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xm87m" Jul 13 17:13:31.657672 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:13:31.657655 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-t2875\"" Jul 13 17:13:31.666832 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:13:31.666811 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xm87m"] Jul 13 17:13:31.747663 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:13:31.747637 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8zrz\" (UniqueName: \"kubernetes.io/projected/9c373c9e-948d-4086-99b2-4d5e2eee22e7-kube-api-access-k8zrz\") pod \"limitador-operator-controller-manager-859d7b78bd-xm87m\" (UID: \"9c373c9e-948d-4086-99b2-4d5e2eee22e7\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xm87m" Jul 13 17:13:31.848610 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:13:31.848586 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8zrz\" (UniqueName: \"kubernetes.io/projected/9c373c9e-948d-4086-99b2-4d5e2eee22e7-kube-api-access-k8zrz\") pod \"limitador-operator-controller-manager-859d7b78bd-xm87m\" (UID: \"9c373c9e-948d-4086-99b2-4d5e2eee22e7\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xm87m" Jul 13 17:13:31.860466 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:13:31.860445 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8zrz\" (UniqueName: \"kubernetes.io/projected/9c373c9e-948d-4086-99b2-4d5e2eee22e7-kube-api-access-k8zrz\") pod \"limitador-operator-controller-manager-859d7b78bd-xm87m\" (UID: \"9c373c9e-948d-4086-99b2-4d5e2eee22e7\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xm87m" Jul 13 17:13:31.965526 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:13:31.965467 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xm87m" Jul 13 17:13:32.101342 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:13:32.101309 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xm87m"] Jul 13 17:13:32.104351 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:13:32.104307 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c373c9e_948d_4086_99b2_4d5e2eee22e7.slice/crio-ea1a06548cd268e83710c038957f2897e0ba38423ac6824077552aadb0cf0c61 WatchSource:0}: Error finding container ea1a06548cd268e83710c038957f2897e0ba38423ac6824077552aadb0cf0c61: Status 404 returned error can't find the container with id ea1a06548cd268e83710c038957f2897e0ba38423ac6824077552aadb0cf0c61 Jul 13 17:13:32.426317 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:13:32.426286 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xm87m" event={"ID":"9c373c9e-948d-4086-99b2-4d5e2eee22e7","Type":"ContainerStarted","Data":"ea1a06548cd268e83710c038957f2897e0ba38423ac6824077552aadb0cf0c61"} Jul 13 17:13:34.435614 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:13:34.435580 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xm87m" event={"ID":"9c373c9e-948d-4086-99b2-4d5e2eee22e7","Type":"ContainerStarted","Data":"695d43c50e7dda4187e9a13fc6a8c5f9f818b412ee01b37158f0eefca8615569"} Jul 13 17:13:34.435952 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:13:34.435723 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xm87m" Jul 13 17:13:34.473710 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:13:34.473662 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xm87m" podStartSLOduration=1.5457750510000001 podStartE2EDuration="3.473646961s" totalImagesPullingTime="1.92787191s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:13:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:13:34.472589078 +0000 UTC m=+510.870247459" watchObservedRunningTime="2026-07-13 17:13:34.473646961 +0000 UTC m=+510.871305340" Jul 13 17:13:41.423342 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:13:41.423310 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-vhrj6" Jul 13 17:13:42.126532 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:13:42.126502 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-bg5np"] Jul 13 17:13:42.129853 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:13:42.129834 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-bg5np" Jul 13 17:13:42.132896 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:13:42.132872 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-jw4k5\"" Jul 13 17:13:42.142679 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:13:42.142657 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-bg5np"] Jul 13 17:13:42.232157 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:13:42.232131 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmb8j\" (UniqueName: \"kubernetes.io/projected/d2a9c410-87d9-4acc-ab9d-3d4ffa017ed5-kube-api-access-wmb8j\") pod \"authorino-operator-75cc8d5cb7-bg5np\" (UID: \"d2a9c410-87d9-4acc-ab9d-3d4ffa017ed5\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-bg5np" Jul 13 17:13:42.332819 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:13:42.332795 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmb8j\" (UniqueName: \"kubernetes.io/projected/d2a9c410-87d9-4acc-ab9d-3d4ffa017ed5-kube-api-access-wmb8j\") pod \"authorino-operator-75cc8d5cb7-bg5np\" (UID: \"d2a9c410-87d9-4acc-ab9d-3d4ffa017ed5\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-bg5np" Jul 13 17:13:42.343513 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:13:42.343488 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmb8j\" (UniqueName: \"kubernetes.io/projected/d2a9c410-87d9-4acc-ab9d-3d4ffa017ed5-kube-api-access-wmb8j\") pod \"authorino-operator-75cc8d5cb7-bg5np\" (UID: \"d2a9c410-87d9-4acc-ab9d-3d4ffa017ed5\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-bg5np" Jul 13 17:13:42.440343 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:13:42.440289 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-bg5np" Jul 13 17:13:42.576641 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:13:42.576613 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-bg5np"] Jul 13 17:13:42.580270 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:13:42.580244 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2a9c410_87d9_4acc_ab9d_3d4ffa017ed5.slice/crio-e87cfc3e16034a05def9ab1845bbaf81ee0b732d5a6cb5ea854274526b3948cb WatchSource:0}: Error finding container e87cfc3e16034a05def9ab1845bbaf81ee0b732d5a6cb5ea854274526b3948cb: Status 404 returned error can't find the container with id e87cfc3e16034a05def9ab1845bbaf81ee0b732d5a6cb5ea854274526b3948cb Jul 13 17:13:43.480700 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:13:43.480637 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-bg5np" event={"ID":"d2a9c410-87d9-4acc-ab9d-3d4ffa017ed5","Type":"ContainerStarted","Data":"e87cfc3e16034a05def9ab1845bbaf81ee0b732d5a6cb5ea854274526b3948cb"} Jul 13 17:13:45.445312 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:13:45.445282 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xm87m" Jul 13 17:13:45.494029 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:13:45.493995 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-bg5np" event={"ID":"d2a9c410-87d9-4acc-ab9d-3d4ffa017ed5","Type":"ContainerStarted","Data":"ca151b13d8672188436f2049dcaf2cedd4b9225cc577bc086728ea7b3cdfec7b"} Jul 13 17:13:45.494208 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:13:45.494110 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-bg5np" Jul 13 17:13:45.512247 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:13:45.512192 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-bg5np" podStartSLOduration=1.198360609 podStartE2EDuration="3.512177138s" totalImagesPullingTime="2.313816529s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:13:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:13:45.510498287 +0000 UTC m=+521.908156667" watchObservedRunningTime="2026-07-13 17:13:45.512177138 +0000 UTC m=+521.909835517" Jul 13 17:13:56.500738 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:13:56.500703 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-bg5np" Jul 13 17:13:59.864357 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:13:59.864325 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xm87m"] Jul 13 17:13:59.864828 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:13:59.864529 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xm87m" podUID="9c373c9e-948d-4086-99b2-4d5e2eee22e7" containerName="manager" containerID="cri-o://695d43c50e7dda4187e9a13fc6a8c5f9f818b412ee01b37158f0eefca8615569" gracePeriod=2 Jul 13 17:13:59.883304 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:13:59.883282 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xm87m"] Jul 13 17:13:59.888325 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:13:59.888290 2569 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"9c373c9e-948d-4086-99b2-4d5e2eee22e7\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-xm87m\": pods \"limitador-operator-controller-manager-859d7b78bd-xm87m\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xm87m" Jul 13 17:13:59.913337 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:13:59.913312 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jxm7m"] Jul 13 17:13:59.913667 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:13:59.913647 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c373c9e-948d-4086-99b2-4d5e2eee22e7" containerName="manager" Jul 13 17:13:59.913667 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:13:59.913663 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c373c9e-948d-4086-99b2-4d5e2eee22e7" containerName="manager" Jul 13 17:13:59.913840 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:13:59.913718 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c373c9e-948d-4086-99b2-4d5e2eee22e7" containerName="manager" Jul 13 17:13:59.916589 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:13:59.916550 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jxm7m" Jul 13 17:13:59.932816 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:13:59.932792 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jxm7m"] Jul 13 17:13:59.952778 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:13:59.952744 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-xm87m\" is forbidden: User \"system:node:ip-10-0-136-220.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-220.ec2.internal' and this object" podUID="9c373c9e-948d-4086-99b2-4d5e2eee22e7" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xm87m" Jul 13 17:14:00.066919 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:14:00.066884 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b49tq\" (UniqueName: \"kubernetes.io/projected/5541c29b-dcd6-4513-9dde-24f95ed624db-kube-api-access-b49tq\") pod \"limitador-operator-controller-manager-859d7b78bd-jxm7m\" (UID: \"5541c29b-dcd6-4513-9dde-24f95ed624db\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jxm7m" Jul 13 17:14:00.105781 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:14:00.105758 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xm87m" Jul 13 17:14:00.107426 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:14:00.107403 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-xm87m\" is forbidden: User \"system:node:ip-10-0-136-220.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-220.ec2.internal' and this object" podUID="9c373c9e-948d-4086-99b2-4d5e2eee22e7" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xm87m" Jul 13 17:14:00.167797 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:14:00.167739 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b49tq\" (UniqueName: \"kubernetes.io/projected/5541c29b-dcd6-4513-9dde-24f95ed624db-kube-api-access-b49tq\") pod \"limitador-operator-controller-manager-859d7b78bd-jxm7m\" (UID: \"5541c29b-dcd6-4513-9dde-24f95ed624db\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jxm7m" Jul 13 17:14:00.193361 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:14:00.193339 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b49tq\" (UniqueName: \"kubernetes.io/projected/5541c29b-dcd6-4513-9dde-24f95ed624db-kube-api-access-b49tq\") pod \"limitador-operator-controller-manager-859d7b78bd-jxm7m\" (UID: \"5541c29b-dcd6-4513-9dde-24f95ed624db\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jxm7m" Jul 13 17:14:00.244711 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:14:00.244687 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jxm7m" Jul 13 17:14:00.269059 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:14:00.269018 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k8zrz\" (UniqueName: \"kubernetes.io/projected/9c373c9e-948d-4086-99b2-4d5e2eee22e7-kube-api-access-k8zrz\") pod \"9c373c9e-948d-4086-99b2-4d5e2eee22e7\" (UID: \"9c373c9e-948d-4086-99b2-4d5e2eee22e7\") " Jul 13 17:14:00.271175 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:14:00.271147 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c373c9e-948d-4086-99b2-4d5e2eee22e7-kube-api-access-k8zrz" (OuterVolumeSpecName: "kube-api-access-k8zrz") pod "9c373c9e-948d-4086-99b2-4d5e2eee22e7" (UID: "9c373c9e-948d-4086-99b2-4d5e2eee22e7"). InnerVolumeSpecName "kube-api-access-k8zrz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:14:00.370768 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:14:00.370739 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k8zrz\" (UniqueName: \"kubernetes.io/projected/9c373c9e-948d-4086-99b2-4d5e2eee22e7-kube-api-access-k8zrz\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:14:00.396167 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:14:00.396143 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jxm7m"] Jul 13 17:14:00.398816 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:14:00.398790 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5541c29b_dcd6_4513_9dde_24f95ed624db.slice/crio-379f4142c689497e9d017fe172893e8781e409b5d05f546f06cf19148aa48c0d WatchSource:0}: Error finding container 379f4142c689497e9d017fe172893e8781e409b5d05f546f06cf19148aa48c0d: Status 404 returned error can't find the container with id 379f4142c689497e9d017fe172893e8781e409b5d05f546f06cf19148aa48c0d Jul 13 17:14:00.563463 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:14:00.563433 2569 generic.go:356] "Generic (PLEG): container finished" podID="9c373c9e-948d-4086-99b2-4d5e2eee22e7" containerID="695d43c50e7dda4187e9a13fc6a8c5f9f818b412ee01b37158f0eefca8615569" exitCode=0 Jul 13 17:14:00.563692 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:14:00.563472 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xm87m" Jul 13 17:14:00.563692 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:14:00.563534 2569 scope.go:117] "RemoveContainer" containerID="695d43c50e7dda4187e9a13fc6a8c5f9f818b412ee01b37158f0eefca8615569" Jul 13 17:14:00.565305 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:14:00.565275 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jxm7m" event={"ID":"5541c29b-dcd6-4513-9dde-24f95ed624db","Type":"ContainerStarted","Data":"2aa6036937bb45302dd314235a4d1ca3b35db1869c3764a1c0760752d5a024c4"} Jul 13 17:14:00.565413 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:14:00.565337 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jxm7m" event={"ID":"5541c29b-dcd6-4513-9dde-24f95ed624db","Type":"ContainerStarted","Data":"379f4142c689497e9d017fe172893e8781e409b5d05f546f06cf19148aa48c0d"} Jul 13 17:14:00.565467 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:14:00.565458 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jxm7m" Jul 13 17:14:00.573698 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:14:00.573476 2569 scope.go:117] "RemoveContainer" containerID="695d43c50e7dda4187e9a13fc6a8c5f9f818b412ee01b37158f0eefca8615569" Jul 13 17:14:00.574860 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:14:00.574834 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"695d43c50e7dda4187e9a13fc6a8c5f9f818b412ee01b37158f0eefca8615569\": container with ID starting with 695d43c50e7dda4187e9a13fc6a8c5f9f818b412ee01b37158f0eefca8615569 not found: ID does not exist" containerID="695d43c50e7dda4187e9a13fc6a8c5f9f818b412ee01b37158f0eefca8615569" Jul 13 17:14:00.574943 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:14:00.574869 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"695d43c50e7dda4187e9a13fc6a8c5f9f818b412ee01b37158f0eefca8615569"} err="failed to get container status \"695d43c50e7dda4187e9a13fc6a8c5f9f818b412ee01b37158f0eefca8615569\": rpc error: code = NotFound desc = could not find container \"695d43c50e7dda4187e9a13fc6a8c5f9f818b412ee01b37158f0eefca8615569\": container with ID starting with 695d43c50e7dda4187e9a13fc6a8c5f9f818b412ee01b37158f0eefca8615569 not found: ID does not exist" Jul 13 17:14:00.597794 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:14:00.597755 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jxm7m" podStartSLOduration=1.5977434019999999 podStartE2EDuration="1.597743402s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:13:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:14:00.595591515 +0000 UTC m=+536.993249914" watchObservedRunningTime="2026-07-13 17:14:00.597743402 +0000 UTC m=+536.995401839" Jul 13 17:14:02.138338 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:14:02.138302 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c373c9e-948d-4086-99b2-4d5e2eee22e7" path="/var/lib/kubelet/pods/9c373c9e-948d-4086-99b2-4d5e2eee22e7/volumes" Jul 13 17:14:11.578255 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:14:11.578225 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jxm7m" Jul 13 17:15:04.058514 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:15:04.058488 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-hzcql_c40e6e88-ab3f-4e3c-a0e7-2400bda333eb/console-operator/1.log" Jul 13 17:15:04.064270 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:15:04.064251 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-hzcql_c40e6e88-ab3f-4e3c-a0e7-2400bda333eb/console-operator/1.log" Jul 13 17:15:04.067221 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:15:04.067199 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q2pnb_6a531b70-4a58-4752-8fa8-5ba634b542ea/ovn-acl-logging/0.log" Jul 13 17:15:04.075795 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:15:04.075777 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q2pnb_6a531b70-4a58-4752-8fa8-5ba634b542ea/ovn-acl-logging/0.log" Jul 13 17:15:29.059849 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:15:29.059759 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-785975bcc4-w2psp"] Jul 13 17:15:29.062932 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:15:29.062908 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-785975bcc4-w2psp" Jul 13 17:15:29.065407 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:15:29.065377 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 13 17:15:29.065520 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:15:29.065445 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 13 17:15:29.065603 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:15:29.065529 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-7hpr6\"" Jul 13 17:15:29.079575 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:15:29.079541 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-785975bcc4-w2psp"] Jul 13 17:15:29.123151 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:15:29.123127 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fbd7ac13-b3aa-40f7-96a3-4ef63d5082e0-webhook-cert\") pod \"maas-controller-785975bcc4-w2psp\" (UID: \"fbd7ac13-b3aa-40f7-96a3-4ef63d5082e0\") " pod="opendatahub/maas-controller-785975bcc4-w2psp" Jul 13 17:15:29.123251 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:15:29.123158 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nwbw\" (UniqueName: \"kubernetes.io/projected/fbd7ac13-b3aa-40f7-96a3-4ef63d5082e0-kube-api-access-9nwbw\") pod \"maas-controller-785975bcc4-w2psp\" (UID: \"fbd7ac13-b3aa-40f7-96a3-4ef63d5082e0\") " pod="opendatahub/maas-controller-785975bcc4-w2psp" Jul 13 17:15:29.223887 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:15:29.223848 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fbd7ac13-b3aa-40f7-96a3-4ef63d5082e0-webhook-cert\") pod \"maas-controller-785975bcc4-w2psp\" (UID: \"fbd7ac13-b3aa-40f7-96a3-4ef63d5082e0\") " pod="opendatahub/maas-controller-785975bcc4-w2psp" Jul 13 17:15:29.224036 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:15:29.224018 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nwbw\" (UniqueName: \"kubernetes.io/projected/fbd7ac13-b3aa-40f7-96a3-4ef63d5082e0-kube-api-access-9nwbw\") pod \"maas-controller-785975bcc4-w2psp\" (UID: \"fbd7ac13-b3aa-40f7-96a3-4ef63d5082e0\") " pod="opendatahub/maas-controller-785975bcc4-w2psp" Jul 13 17:15:29.226436 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:15:29.226411 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fbd7ac13-b3aa-40f7-96a3-4ef63d5082e0-webhook-cert\") pod \"maas-controller-785975bcc4-w2psp\" (UID: \"fbd7ac13-b3aa-40f7-96a3-4ef63d5082e0\") " pod="opendatahub/maas-controller-785975bcc4-w2psp" Jul 13 17:15:29.231766 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:15:29.231745 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nwbw\" (UniqueName: \"kubernetes.io/projected/fbd7ac13-b3aa-40f7-96a3-4ef63d5082e0-kube-api-access-9nwbw\") pod \"maas-controller-785975bcc4-w2psp\" (UID: \"fbd7ac13-b3aa-40f7-96a3-4ef63d5082e0\") " pod="opendatahub/maas-controller-785975bcc4-w2psp" Jul 13 17:15:29.251677 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:15:29.251652 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-db977844c-9jf6c"] Jul 13 17:15:29.254499 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:15:29.254482 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-db977844c-9jf6c" Jul 13 17:15:29.263552 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:15:29.263531 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-db977844c-9jf6c"] Jul 13 17:15:29.324670 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:15:29.324615 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92mcv\" (UniqueName: \"kubernetes.io/projected/12352f14-702e-45f7-81d8-7cb31d934475-kube-api-access-92mcv\") pod \"maas-controller-db977844c-9jf6c\" (UID: \"12352f14-702e-45f7-81d8-7cb31d934475\") " pod="opendatahub/maas-controller-db977844c-9jf6c" Jul 13 17:15:29.324670 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:15:29.324643 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/12352f14-702e-45f7-81d8-7cb31d934475-webhook-cert\") pod \"maas-controller-db977844c-9jf6c\" (UID: \"12352f14-702e-45f7-81d8-7cb31d934475\") " pod="opendatahub/maas-controller-db977844c-9jf6c" Jul 13 17:15:29.373137 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:15:29.373111 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-785975bcc4-w2psp" Jul 13 17:15:29.425809 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:15:29.425773 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92mcv\" (UniqueName: \"kubernetes.io/projected/12352f14-702e-45f7-81d8-7cb31d934475-kube-api-access-92mcv\") pod \"maas-controller-db977844c-9jf6c\" (UID: \"12352f14-702e-45f7-81d8-7cb31d934475\") " pod="opendatahub/maas-controller-db977844c-9jf6c" Jul 13 17:15:29.425935 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:15:29.425830 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/12352f14-702e-45f7-81d8-7cb31d934475-webhook-cert\") pod \"maas-controller-db977844c-9jf6c\" (UID: \"12352f14-702e-45f7-81d8-7cb31d934475\") " pod="opendatahub/maas-controller-db977844c-9jf6c" Jul 13 17:15:29.428238 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:15:29.428215 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/12352f14-702e-45f7-81d8-7cb31d934475-webhook-cert\") pod \"maas-controller-db977844c-9jf6c\" (UID: \"12352f14-702e-45f7-81d8-7cb31d934475\") " pod="opendatahub/maas-controller-db977844c-9jf6c" Jul 13 17:15:29.435715 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:15:29.435662 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-92mcv\" (UniqueName: \"kubernetes.io/projected/12352f14-702e-45f7-81d8-7cb31d934475-kube-api-access-92mcv\") pod \"maas-controller-db977844c-9jf6c\" (UID: \"12352f14-702e-45f7-81d8-7cb31d934475\") " pod="opendatahub/maas-controller-db977844c-9jf6c" Jul 13 17:15:29.502546 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:15:29.502522 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-785975bcc4-w2psp"] Jul 13 17:15:29.504615 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:15:29.504585 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbd7ac13_b3aa_40f7_96a3_4ef63d5082e0.slice/crio-e0dba9613f1ee795a770be00cc0cf6265927535522f183aa6a357e661abb0763 WatchSource:0}: Error finding container e0dba9613f1ee795a770be00cc0cf6265927535522f183aa6a357e661abb0763: Status 404 returned error can't find the container with id e0dba9613f1ee795a770be00cc0cf6265927535522f183aa6a357e661abb0763 Jul 13 17:15:29.565594 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:15:29.565553 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-db977844c-9jf6c" Jul 13 17:15:29.695321 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:15:29.695305 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-db977844c-9jf6c"] Jul 13 17:15:29.697415 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:15:29.697386 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12352f14_702e_45f7_81d8_7cb31d934475.slice/crio-ab0a27c4b8a85149cedc853285a5fb42a3b271e7c09c1f8942cf3d87809ab1a7 WatchSource:0}: Error finding container ab0a27c4b8a85149cedc853285a5fb42a3b271e7c09c1f8942cf3d87809ab1a7: Status 404 returned error can't find the container with id ab0a27c4b8a85149cedc853285a5fb42a3b271e7c09c1f8942cf3d87809ab1a7 Jul 13 17:15:29.958430 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:15:29.958356 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-db977844c-9jf6c" event={"ID":"12352f14-702e-45f7-81d8-7cb31d934475","Type":"ContainerStarted","Data":"ab0a27c4b8a85149cedc853285a5fb42a3b271e7c09c1f8942cf3d87809ab1a7"} Jul 13 17:15:29.959509 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:15:29.959486 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-785975bcc4-w2psp" event={"ID":"fbd7ac13-b3aa-40f7-96a3-4ef63d5082e0","Type":"ContainerStarted","Data":"e0dba9613f1ee795a770be00cc0cf6265927535522f183aa6a357e661abb0763"} Jul 13 17:15:32.977892 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:15:32.977851 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-db977844c-9jf6c" event={"ID":"12352f14-702e-45f7-81d8-7cb31d934475","Type":"ContainerStarted","Data":"61bd65545f79966e5809c3a5b18ca1f7ac344eb0bdaaa0cdfdda38af4241af03"} Jul 13 17:15:32.978312 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:15:32.977937 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-db977844c-9jf6c" Jul 13 17:15:32.979406 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:15:32.979382 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-785975bcc4-w2psp" event={"ID":"fbd7ac13-b3aa-40f7-96a3-4ef63d5082e0","Type":"ContainerStarted","Data":"3f0cab6511fa5e76cb0b158e0755a49ef13bcd7b5f112d7d2cb00a43149e51e2"} Jul 13 17:15:32.979523 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:15:32.979440 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-785975bcc4-w2psp" Jul 13 17:15:32.994783 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:15:32.994744 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-db977844c-9jf6c" podStartSLOduration=1.279069458 podStartE2EDuration="3.994732088s" totalImagesPullingTime="2.71566263s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:15:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:15:32.992512396 +0000 UTC m=+629.390170785" watchObservedRunningTime="2026-07-13 17:15:32.994732088 +0000 UTC m=+629.392390445" Jul 13 17:15:33.009180 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:15:33.009141 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-785975bcc4-w2psp" podStartSLOduration=1.091085221 podStartE2EDuration="4.009129266s" totalImagesPullingTime="2.918044045s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:15:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:15:33.00767785 +0000 UTC m=+629.405336232" watchObservedRunningTime="2026-07-13 17:15:33.009129266 +0000 UTC m=+629.406787645" Jul 13 17:15:43.986337 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:15:43.986307 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-785975bcc4-w2psp" Jul 13 17:15:43.986725 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:15:43.986376 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-db977844c-9jf6c" Jul 13 17:15:44.123806 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:15:44.123773 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-785975bcc4-w2psp"] Jul 13 17:15:44.124035 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:15:44.124008 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-785975bcc4-w2psp" podUID="fbd7ac13-b3aa-40f7-96a3-4ef63d5082e0" containerName="manager" containerID="cri-o://3f0cab6511fa5e76cb0b158e0755a49ef13bcd7b5f112d7d2cb00a43149e51e2" gracePeriod=30 Jul 13 17:15:44.372521 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:15:44.372492 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-785975bcc4-w2psp" Jul 13 17:15:44.446815 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:15:44.446788 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9nwbw\" (UniqueName: \"kubernetes.io/projected/fbd7ac13-b3aa-40f7-96a3-4ef63d5082e0-kube-api-access-9nwbw\") pod \"fbd7ac13-b3aa-40f7-96a3-4ef63d5082e0\" (UID: \"fbd7ac13-b3aa-40f7-96a3-4ef63d5082e0\") " Jul 13 17:15:44.446932 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:15:44.446894 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fbd7ac13-b3aa-40f7-96a3-4ef63d5082e0-webhook-cert\") pod \"fbd7ac13-b3aa-40f7-96a3-4ef63d5082e0\" (UID: \"fbd7ac13-b3aa-40f7-96a3-4ef63d5082e0\") " Jul 13 17:15:44.448860 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:15:44.448833 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbd7ac13-b3aa-40f7-96a3-4ef63d5082e0-kube-api-access-9nwbw" (OuterVolumeSpecName: "kube-api-access-9nwbw") pod "fbd7ac13-b3aa-40f7-96a3-4ef63d5082e0" (UID: "fbd7ac13-b3aa-40f7-96a3-4ef63d5082e0"). InnerVolumeSpecName "kube-api-access-9nwbw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:15:44.448964 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:15:44.448889 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fbd7ac13-b3aa-40f7-96a3-4ef63d5082e0-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "fbd7ac13-b3aa-40f7-96a3-4ef63d5082e0" (UID: "fbd7ac13-b3aa-40f7-96a3-4ef63d5082e0"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 17:15:44.547434 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:15:44.547412 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9nwbw\" (UniqueName: \"kubernetes.io/projected/fbd7ac13-b3aa-40f7-96a3-4ef63d5082e0-kube-api-access-9nwbw\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:15:44.547434 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:15:44.547434 2569 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fbd7ac13-b3aa-40f7-96a3-4ef63d5082e0-webhook-cert\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:15:45.035322 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:15:45.035292 2569 generic.go:356] "Generic (PLEG): container finished" podID="fbd7ac13-b3aa-40f7-96a3-4ef63d5082e0" containerID="3f0cab6511fa5e76cb0b158e0755a49ef13bcd7b5f112d7d2cb00a43149e51e2" exitCode=0 Jul 13 17:15:45.035718 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:15:45.035334 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-785975bcc4-w2psp" Jul 13 17:15:45.035718 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:15:45.035346 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-785975bcc4-w2psp" event={"ID":"fbd7ac13-b3aa-40f7-96a3-4ef63d5082e0","Type":"ContainerDied","Data":"3f0cab6511fa5e76cb0b158e0755a49ef13bcd7b5f112d7d2cb00a43149e51e2"} Jul 13 17:15:45.035718 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:15:45.035418 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-785975bcc4-w2psp" event={"ID":"fbd7ac13-b3aa-40f7-96a3-4ef63d5082e0","Type":"ContainerDied","Data":"e0dba9613f1ee795a770be00cc0cf6265927535522f183aa6a357e661abb0763"} Jul 13 17:15:45.035718 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:15:45.035444 2569 scope.go:117] "RemoveContainer" containerID="3f0cab6511fa5e76cb0b158e0755a49ef13bcd7b5f112d7d2cb00a43149e51e2" Jul 13 17:15:45.044638 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:15:45.044616 2569 scope.go:117] "RemoveContainer" containerID="3f0cab6511fa5e76cb0b158e0755a49ef13bcd7b5f112d7d2cb00a43149e51e2" Jul 13 17:15:45.044884 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:15:45.044865 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f0cab6511fa5e76cb0b158e0755a49ef13bcd7b5f112d7d2cb00a43149e51e2\": container with ID starting with 3f0cab6511fa5e76cb0b158e0755a49ef13bcd7b5f112d7d2cb00a43149e51e2 not found: ID does not exist" containerID="3f0cab6511fa5e76cb0b158e0755a49ef13bcd7b5f112d7d2cb00a43149e51e2" Jul 13 17:15:45.044925 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:15:45.044893 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f0cab6511fa5e76cb0b158e0755a49ef13bcd7b5f112d7d2cb00a43149e51e2"} err="failed to get container status \"3f0cab6511fa5e76cb0b158e0755a49ef13bcd7b5f112d7d2cb00a43149e51e2\": rpc error: code = NotFound desc = could not find container \"3f0cab6511fa5e76cb0b158e0755a49ef13bcd7b5f112d7d2cb00a43149e51e2\": container with ID starting with 3f0cab6511fa5e76cb0b158e0755a49ef13bcd7b5f112d7d2cb00a43149e51e2 not found: ID does not exist" Jul 13 17:15:45.055927 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:15:45.055896 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-785975bcc4-w2psp"] Jul 13 17:15:45.059681 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:15:45.059661 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-785975bcc4-w2psp"] Jul 13 17:15:46.135903 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:15:46.135871 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fbd7ac13-b3aa-40f7-96a3-4ef63d5082e0" path="/var/lib/kubelet/pods/fbd7ac13-b3aa-40f7-96a3-4ef63d5082e0/volumes" Jul 13 17:16:07.500981 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:16:07.500950 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-t2l47"] Jul 13 17:16:07.501358 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:16:07.501316 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fbd7ac13-b3aa-40f7-96a3-4ef63d5082e0" containerName="manager" Jul 13 17:16:07.501358 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:16:07.501329 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbd7ac13-b3aa-40f7-96a3-4ef63d5082e0" containerName="manager" Jul 13 17:16:07.501444 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:16:07.501414 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="fbd7ac13-b3aa-40f7-96a3-4ef63d5082e0" containerName="manager" Jul 13 17:16:07.505805 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:16:07.505784 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-t2l47" Jul 13 17:16:07.508102 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:16:07.508073 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 13 17:16:07.508228 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:16:07.508176 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-27s9c\"" Jul 13 17:16:07.513517 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:16:07.513491 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-t2l47"] Jul 13 17:16:07.614995 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:16:07.614964 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e32dc0d2-f8c6-4ca8-a90f-ae59ff574fa1-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-t2l47\" (UID: \"e32dc0d2-f8c6-4ca8-a90f-ae59ff574fa1\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-t2l47" Jul 13 17:16:07.615158 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:16:07.615009 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47m2n\" (UniqueName: \"kubernetes.io/projected/e32dc0d2-f8c6-4ca8-a90f-ae59ff574fa1-kube-api-access-47m2n\") pod \"payload-processing-f9b8d9b9b-t2l47\" (UID: \"e32dc0d2-f8c6-4ca8-a90f-ae59ff574fa1\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-t2l47" Jul 13 17:16:07.715633 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:16:07.715603 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e32dc0d2-f8c6-4ca8-a90f-ae59ff574fa1-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-t2l47\" (UID: \"e32dc0d2-f8c6-4ca8-a90f-ae59ff574fa1\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-t2l47" Jul 13 17:16:07.715790 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:16:07.715654 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47m2n\" (UniqueName: \"kubernetes.io/projected/e32dc0d2-f8c6-4ca8-a90f-ae59ff574fa1-kube-api-access-47m2n\") pod \"payload-processing-f9b8d9b9b-t2l47\" (UID: \"e32dc0d2-f8c6-4ca8-a90f-ae59ff574fa1\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-t2l47" Jul 13 17:16:07.716321 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:16:07.716297 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e32dc0d2-f8c6-4ca8-a90f-ae59ff574fa1-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-t2l47\" (UID: \"e32dc0d2-f8c6-4ca8-a90f-ae59ff574fa1\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-t2l47" Jul 13 17:16:07.726141 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:16:07.726113 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47m2n\" (UniqueName: \"kubernetes.io/projected/e32dc0d2-f8c6-4ca8-a90f-ae59ff574fa1-kube-api-access-47m2n\") pod \"payload-processing-f9b8d9b9b-t2l47\" (UID: \"e32dc0d2-f8c6-4ca8-a90f-ae59ff574fa1\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-t2l47" Jul 13 17:16:07.816887 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:16:07.816863 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-t2l47" Jul 13 17:16:07.946988 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:16:07.946958 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-t2l47"] Jul 13 17:16:07.948807 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:16:07.948779 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode32dc0d2_f8c6_4ca8_a90f_ae59ff574fa1.slice/crio-dd9cafda8c2d16ad52236a769470a5299dcaa2ed89d021f90cb1db022f1c23c3 WatchSource:0}: Error finding container dd9cafda8c2d16ad52236a769470a5299dcaa2ed89d021f90cb1db022f1c23c3: Status 404 returned error can't find the container with id dd9cafda8c2d16ad52236a769470a5299dcaa2ed89d021f90cb1db022f1c23c3 Jul 13 17:16:08.139336 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:16:08.139257 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-t2l47" event={"ID":"e32dc0d2-f8c6-4ca8-a90f-ae59ff574fa1","Type":"ContainerStarted","Data":"dd9cafda8c2d16ad52236a769470a5299dcaa2ed89d021f90cb1db022f1c23c3"} Jul 13 17:16:10.151662 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:16:10.151624 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-t2l47" event={"ID":"e32dc0d2-f8c6-4ca8-a90f-ae59ff574fa1","Type":"ContainerStarted","Data":"d2ad31a9e229fc961d1821635030b451549a63ac1a1c9b472b905f24c745e814"} Jul 13 17:16:10.152113 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:16:10.151755 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-t2l47" Jul 13 17:16:10.167965 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:16:10.167915 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-f9b8d9b9b-t2l47" podStartSLOduration=1.88731241 podStartE2EDuration="3.167900125s" totalImagesPullingTime="1.280587715s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:16:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:16:10.166645947 +0000 UTC m=+666.564304347" watchObservedRunningTime="2026-07-13 17:16:10.167900125 +0000 UTC m=+666.565558504" Jul 13 17:16:21.158272 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:16:21.158240 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-t2l47" Jul 13 17:16:22.326323 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:16:22.326287 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ntcvc"] Jul 13 17:16:22.328952 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:16:22.328929 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ntcvc" Jul 13 17:16:22.331472 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:16:22.331449 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-jcvk5\"" Jul 13 17:16:22.331755 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:16:22.331734 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 13 17:16:22.331932 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:16:22.331915 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 13 17:16:22.332498 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:16:22.332481 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 13 17:16:22.341750 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:16:22.341729 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ntcvc"] Jul 13 17:16:22.436677 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:16:22.436647 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/895459f5-1d2d-411c-8aa1-357bc8116f85-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ntcvc\" (UID: \"895459f5-1d2d-411c-8aa1-357bc8116f85\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ntcvc" Jul 13 17:16:22.436860 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:16:22.436842 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/895459f5-1d2d-411c-8aa1-357bc8116f85-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ntcvc\" (UID: \"895459f5-1d2d-411c-8aa1-357bc8116f85\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ntcvc" Jul 13 17:16:22.437000 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:16:22.436983 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/895459f5-1d2d-411c-8aa1-357bc8116f85-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ntcvc\" (UID: \"895459f5-1d2d-411c-8aa1-357bc8116f85\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ntcvc" Jul 13 17:16:22.437128 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:16:22.437114 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/895459f5-1d2d-411c-8aa1-357bc8116f85-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ntcvc\" (UID: \"895459f5-1d2d-411c-8aa1-357bc8116f85\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ntcvc" Jul 13 17:16:22.437240 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:16:22.437225 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lnf6\" (UniqueName: \"kubernetes.io/projected/895459f5-1d2d-411c-8aa1-357bc8116f85-kube-api-access-2lnf6\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ntcvc\" (UID: \"895459f5-1d2d-411c-8aa1-357bc8116f85\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ntcvc" Jul 13 17:16:22.437342 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:16:22.437328 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/895459f5-1d2d-411c-8aa1-357bc8116f85-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ntcvc\" (UID: \"895459f5-1d2d-411c-8aa1-357bc8116f85\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ntcvc" Jul 13 17:16:22.538068 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:16:22.538041 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/895459f5-1d2d-411c-8aa1-357bc8116f85-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ntcvc\" (UID: \"895459f5-1d2d-411c-8aa1-357bc8116f85\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ntcvc" Jul 13 17:16:22.538068 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:16:22.538072 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lnf6\" (UniqueName: \"kubernetes.io/projected/895459f5-1d2d-411c-8aa1-357bc8116f85-kube-api-access-2lnf6\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ntcvc\" (UID: \"895459f5-1d2d-411c-8aa1-357bc8116f85\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ntcvc" Jul 13 17:16:22.538241 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:16:22.538090 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/895459f5-1d2d-411c-8aa1-357bc8116f85-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ntcvc\" (UID: \"895459f5-1d2d-411c-8aa1-357bc8116f85\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ntcvc" Jul 13 17:16:22.538241 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:16:22.538148 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/895459f5-1d2d-411c-8aa1-357bc8116f85-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ntcvc\" (UID: \"895459f5-1d2d-411c-8aa1-357bc8116f85\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ntcvc" Jul 13 17:16:22.538351 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:16:22.538295 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/895459f5-1d2d-411c-8aa1-357bc8116f85-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ntcvc\" (UID: \"895459f5-1d2d-411c-8aa1-357bc8116f85\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ntcvc" Jul 13 17:16:22.538351 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:16:22.538337 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/895459f5-1d2d-411c-8aa1-357bc8116f85-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ntcvc\" (UID: \"895459f5-1d2d-411c-8aa1-357bc8116f85\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ntcvc" Jul 13 17:16:22.538453 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:16:22.538405 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/895459f5-1d2d-411c-8aa1-357bc8116f85-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ntcvc\" (UID: \"895459f5-1d2d-411c-8aa1-357bc8116f85\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ntcvc" Jul 13 17:16:22.538609 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:16:22.538594 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/895459f5-1d2d-411c-8aa1-357bc8116f85-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ntcvc\" (UID: \"895459f5-1d2d-411c-8aa1-357bc8116f85\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ntcvc" Jul 13 17:16:22.538659 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:16:22.538646 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/895459f5-1d2d-411c-8aa1-357bc8116f85-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ntcvc\" (UID: \"895459f5-1d2d-411c-8aa1-357bc8116f85\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ntcvc" Jul 13 17:16:22.540411 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:16:22.540386 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/895459f5-1d2d-411c-8aa1-357bc8116f85-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ntcvc\" (UID: \"895459f5-1d2d-411c-8aa1-357bc8116f85\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ntcvc" Jul 13 17:16:22.540497 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:16:22.540479 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/895459f5-1d2d-411c-8aa1-357bc8116f85-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ntcvc\" (UID: \"895459f5-1d2d-411c-8aa1-357bc8116f85\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ntcvc" Jul 13 17:16:22.547219 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:16:22.547199 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lnf6\" (UniqueName: \"kubernetes.io/projected/895459f5-1d2d-411c-8aa1-357bc8116f85-kube-api-access-2lnf6\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ntcvc\" (UID: \"895459f5-1d2d-411c-8aa1-357bc8116f85\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ntcvc" Jul 13 17:16:22.639749 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:16:22.639691 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ntcvc" Jul 13 17:16:22.769986 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:16:22.769962 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ntcvc"] Jul 13 17:16:22.772099 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:16:22.772068 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod895459f5_1d2d_411c_8aa1_357bc8116f85.slice/crio-aedfeea28102a0fb238bf1c553b0a8ed6a4cc9c99f56263a0fcdcc8efded88f8 WatchSource:0}: Error finding container aedfeea28102a0fb238bf1c553b0a8ed6a4cc9c99f56263a0fcdcc8efded88f8: Status 404 returned error can't find the container with id aedfeea28102a0fb238bf1c553b0a8ed6a4cc9c99f56263a0fcdcc8efded88f8 Jul 13 17:16:23.211787 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:16:23.211746 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ntcvc" event={"ID":"895459f5-1d2d-411c-8aa1-357bc8116f85","Type":"ContainerStarted","Data":"aedfeea28102a0fb238bf1c553b0a8ed6a4cc9c99f56263a0fcdcc8efded88f8"} Jul 13 17:16:24.217329 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:16:24.217237 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ntcvc" event={"ID":"895459f5-1d2d-411c-8aa1-357bc8116f85","Type":"ContainerStarted","Data":"c95cecb228c1ac810dd3ec52a01e8494466997010d548cc27f9a6be27a10a081"} Jul 13 17:16:24.234838 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:16:24.234792 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ntcvc" podStartSLOduration=1.145193478 podStartE2EDuration="2.234778278s" totalImagesPullingTime="1.0895848s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:16:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:16:24.232928903 +0000 UTC m=+680.630587281" watchObservedRunningTime="2026-07-13 17:16:24.234778278 +0000 UTC m=+680.632436656" Jul 13 17:16:32.640138 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:16:32.640101 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ntcvc" Jul 13 17:16:32.640514 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:16:32.640158 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ntcvc" Jul 13 17:16:32.652669 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:16:32.652646 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ntcvc" Jul 13 17:16:33.267228 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:16:33.267199 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ntcvc" Jul 13 17:17:30.425999 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:30.425921 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-49fj9"] Jul 13 17:17:30.429140 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:30.429123 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-49fj9" Jul 13 17:17:30.431322 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:30.431299 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 13 17:17:30.437363 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:30.437338 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-49fj9"] Jul 13 17:17:30.449251 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:30.449230 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db3344e0-315b-48f3-96f5-543fdfdf024b-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-49fj9\" (UID: \"db3344e0-315b-48f3-96f5-543fdfdf024b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-49fj9" Jul 13 17:17:30.449367 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:30.449273 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db3344e0-315b-48f3-96f5-543fdfdf024b-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-49fj9\" (UID: \"db3344e0-315b-48f3-96f5-543fdfdf024b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-49fj9" Jul 13 17:17:30.449437 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:30.449370 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db3344e0-315b-48f3-96f5-543fdfdf024b-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-49fj9\" (UID: \"db3344e0-315b-48f3-96f5-543fdfdf024b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-49fj9" Jul 13 17:17:30.449437 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:30.449406 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db3344e0-315b-48f3-96f5-543fdfdf024b-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-49fj9\" (UID: \"db3344e0-315b-48f3-96f5-543fdfdf024b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-49fj9" Jul 13 17:17:30.449520 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:30.449491 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xwz7\" (UniqueName: \"kubernetes.io/projected/db3344e0-315b-48f3-96f5-543fdfdf024b-kube-api-access-8xwz7\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-49fj9\" (UID: \"db3344e0-315b-48f3-96f5-543fdfdf024b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-49fj9" Jul 13 17:17:30.449588 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:30.449553 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db3344e0-315b-48f3-96f5-543fdfdf024b-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-49fj9\" (UID: \"db3344e0-315b-48f3-96f5-543fdfdf024b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-49fj9" Jul 13 17:17:30.549968 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:30.549939 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db3344e0-315b-48f3-96f5-543fdfdf024b-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-49fj9\" (UID: \"db3344e0-315b-48f3-96f5-543fdfdf024b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-49fj9" Jul 13 17:17:30.549968 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:30.549970 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db3344e0-315b-48f3-96f5-543fdfdf024b-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-49fj9\" (UID: \"db3344e0-315b-48f3-96f5-543fdfdf024b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-49fj9" Jul 13 17:17:30.550240 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:30.549995 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xwz7\" (UniqueName: \"kubernetes.io/projected/db3344e0-315b-48f3-96f5-543fdfdf024b-kube-api-access-8xwz7\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-49fj9\" (UID: \"db3344e0-315b-48f3-96f5-543fdfdf024b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-49fj9" Jul 13 17:17:30.550240 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:30.550037 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db3344e0-315b-48f3-96f5-543fdfdf024b-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-49fj9\" (UID: \"db3344e0-315b-48f3-96f5-543fdfdf024b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-49fj9" Jul 13 17:17:30.550240 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:30.550082 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db3344e0-315b-48f3-96f5-543fdfdf024b-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-49fj9\" (UID: \"db3344e0-315b-48f3-96f5-543fdfdf024b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-49fj9" Jul 13 17:17:30.550240 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:30.550112 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db3344e0-315b-48f3-96f5-543fdfdf024b-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-49fj9\" (UID: \"db3344e0-315b-48f3-96f5-543fdfdf024b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-49fj9" Jul 13 17:17:30.550481 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:30.550404 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db3344e0-315b-48f3-96f5-543fdfdf024b-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-49fj9\" (UID: \"db3344e0-315b-48f3-96f5-543fdfdf024b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-49fj9" Jul 13 17:17:30.550481 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:30.550457 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db3344e0-315b-48f3-96f5-543fdfdf024b-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-49fj9\" (UID: \"db3344e0-315b-48f3-96f5-543fdfdf024b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-49fj9" Jul 13 17:17:30.550590 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:30.550488 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db3344e0-315b-48f3-96f5-543fdfdf024b-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-49fj9\" (UID: \"db3344e0-315b-48f3-96f5-543fdfdf024b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-49fj9" Jul 13 17:17:30.552294 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:30.552269 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db3344e0-315b-48f3-96f5-543fdfdf024b-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-49fj9\" (UID: \"db3344e0-315b-48f3-96f5-543fdfdf024b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-49fj9" Jul 13 17:17:30.552812 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:30.552790 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db3344e0-315b-48f3-96f5-543fdfdf024b-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-49fj9\" (UID: \"db3344e0-315b-48f3-96f5-543fdfdf024b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-49fj9" Jul 13 17:17:30.557261 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:30.557243 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xwz7\" (UniqueName: \"kubernetes.io/projected/db3344e0-315b-48f3-96f5-543fdfdf024b-kube-api-access-8xwz7\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-49fj9\" (UID: \"db3344e0-315b-48f3-96f5-543fdfdf024b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-49fj9" Jul 13 17:17:30.739451 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:30.739373 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-49fj9" Jul 13 17:17:30.872040 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:30.872016 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-49fj9"] Jul 13 17:17:30.874236 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:17:30.874205 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb3344e0_315b_48f3_96f5_543fdfdf024b.slice/crio-06330cdd65bf6431562eb8e477eba6a7ad4b5e2c72780359423101c45f9762c6 WatchSource:0}: Error finding container 06330cdd65bf6431562eb8e477eba6a7ad4b5e2c72780359423101c45f9762c6: Status 404 returned error can't find the container with id 06330cdd65bf6431562eb8e477eba6a7ad4b5e2c72780359423101c45f9762c6 Jul 13 17:17:30.875701 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:30.875680 2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 17:17:31.520517 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:31.520434 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-49fj9" event={"ID":"db3344e0-315b-48f3-96f5-543fdfdf024b","Type":"ContainerStarted","Data":"3be8d51224d47a67e5c6601fc22403b198c91abb5af6ae3fe9b6cea36691ea1e"} Jul 13 17:17:31.520517 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:31.520490 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-49fj9" event={"ID":"db3344e0-315b-48f3-96f5-543fdfdf024b","Type":"ContainerStarted","Data":"06330cdd65bf6431562eb8e477eba6a7ad4b5e2c72780359423101c45f9762c6"} Jul 13 17:17:31.537531 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:31.537486 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-49fj9" podStartSLOduration=1.324545838 podStartE2EDuration="1.53747122s" totalImagesPullingTime="212.925382ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:17:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:17:31.535909229 +0000 UTC m=+747.933567608" watchObservedRunningTime="2026-07-13 17:17:31.53747122 +0000 UTC m=+747.935129635" Jul 13 17:17:35.115962 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:35.115925 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vnw7h"] Jul 13 17:17:35.119364 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:35.119346 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vnw7h" Jul 13 17:17:35.121338 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:35.121312 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 13 17:17:35.127308 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:35.127277 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vnw7h"] Jul 13 17:17:35.189548 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:35.189512 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd482960-43cd-4e01-89a3-12e37fdf44c9-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vnw7h\" (UID: \"bd482960-43cd-4e01-89a3-12e37fdf44c9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vnw7h" Jul 13 17:17:35.189691 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:35.189577 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5xmc\" (UniqueName: \"kubernetes.io/projected/bd482960-43cd-4e01-89a3-12e37fdf44c9-kube-api-access-p5xmc\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vnw7h\" (UID: \"bd482960-43cd-4e01-89a3-12e37fdf44c9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vnw7h" Jul 13 17:17:35.189691 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:35.189627 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd482960-43cd-4e01-89a3-12e37fdf44c9-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vnw7h\" (UID: \"bd482960-43cd-4e01-89a3-12e37fdf44c9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vnw7h" Jul 13 17:17:35.189691 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:35.189661 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd482960-43cd-4e01-89a3-12e37fdf44c9-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vnw7h\" (UID: \"bd482960-43cd-4e01-89a3-12e37fdf44c9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vnw7h" Jul 13 17:17:35.189846 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:35.189708 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd482960-43cd-4e01-89a3-12e37fdf44c9-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vnw7h\" (UID: \"bd482960-43cd-4e01-89a3-12e37fdf44c9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vnw7h" Jul 13 17:17:35.189846 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:35.189786 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd482960-43cd-4e01-89a3-12e37fdf44c9-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vnw7h\" (UID: \"bd482960-43cd-4e01-89a3-12e37fdf44c9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vnw7h" Jul 13 17:17:35.290259 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:35.290226 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd482960-43cd-4e01-89a3-12e37fdf44c9-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vnw7h\" (UID: \"bd482960-43cd-4e01-89a3-12e37fdf44c9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vnw7h" Jul 13 17:17:35.290417 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:35.290292 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd482960-43cd-4e01-89a3-12e37fdf44c9-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vnw7h\" (UID: \"bd482960-43cd-4e01-89a3-12e37fdf44c9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vnw7h" Jul 13 17:17:35.290417 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:35.290323 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5xmc\" (UniqueName: \"kubernetes.io/projected/bd482960-43cd-4e01-89a3-12e37fdf44c9-kube-api-access-p5xmc\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vnw7h\" (UID: \"bd482960-43cd-4e01-89a3-12e37fdf44c9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vnw7h" Jul 13 17:17:35.290417 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:35.290367 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd482960-43cd-4e01-89a3-12e37fdf44c9-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vnw7h\" (UID: \"bd482960-43cd-4e01-89a3-12e37fdf44c9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vnw7h" Jul 13 17:17:35.290417 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:35.290396 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd482960-43cd-4e01-89a3-12e37fdf44c9-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vnw7h\" (UID: \"bd482960-43cd-4e01-89a3-12e37fdf44c9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vnw7h" Jul 13 17:17:35.290670 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:35.290426 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd482960-43cd-4e01-89a3-12e37fdf44c9-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vnw7h\" (UID: \"bd482960-43cd-4e01-89a3-12e37fdf44c9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vnw7h" Jul 13 17:17:35.290728 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:35.290673 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd482960-43cd-4e01-89a3-12e37fdf44c9-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vnw7h\" (UID: \"bd482960-43cd-4e01-89a3-12e37fdf44c9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vnw7h" Jul 13 17:17:35.290832 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:35.290808 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd482960-43cd-4e01-89a3-12e37fdf44c9-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vnw7h\" (UID: \"bd482960-43cd-4e01-89a3-12e37fdf44c9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vnw7h" Jul 13 17:17:35.290919 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:35.290812 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd482960-43cd-4e01-89a3-12e37fdf44c9-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vnw7h\" (UID: \"bd482960-43cd-4e01-89a3-12e37fdf44c9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vnw7h" Jul 13 17:17:35.292604 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:35.292587 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd482960-43cd-4e01-89a3-12e37fdf44c9-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vnw7h\" (UID: \"bd482960-43cd-4e01-89a3-12e37fdf44c9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vnw7h" Jul 13 17:17:35.292790 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:35.292773 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd482960-43cd-4e01-89a3-12e37fdf44c9-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vnw7h\" (UID: \"bd482960-43cd-4e01-89a3-12e37fdf44c9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vnw7h" Jul 13 17:17:35.297639 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:35.297616 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5xmc\" (UniqueName: \"kubernetes.io/projected/bd482960-43cd-4e01-89a3-12e37fdf44c9-kube-api-access-p5xmc\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vnw7h\" (UID: \"bd482960-43cd-4e01-89a3-12e37fdf44c9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vnw7h" Jul 13 17:17:35.429271 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:35.429210 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vnw7h" Jul 13 17:17:35.558399 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:35.558375 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vnw7h"] Jul 13 17:17:35.560873 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:17:35.560843 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbd482960_43cd_4e01_89a3_12e37fdf44c9.slice/crio-0d812e62d22796b2bdc9a22f25161898c776c5874b72ac0a27d5d2e361245b79 WatchSource:0}: Error finding container 0d812e62d22796b2bdc9a22f25161898c776c5874b72ac0a27d5d2e361245b79: Status 404 returned error can't find the container with id 0d812e62d22796b2bdc9a22f25161898c776c5874b72ac0a27d5d2e361245b79 Jul 13 17:17:36.545472 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:36.545437 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vnw7h" event={"ID":"bd482960-43cd-4e01-89a3-12e37fdf44c9","Type":"ContainerStarted","Data":"c2af4558c0750006c46daf99ccade0c346dd2140d78b982987906e8582e33d04"} Jul 13 17:17:36.545472 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:36.545476 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vnw7h" event={"ID":"bd482960-43cd-4e01-89a3-12e37fdf44c9","Type":"ContainerStarted","Data":"0d812e62d22796b2bdc9a22f25161898c776c5874b72ac0a27d5d2e361245b79"} Jul 13 17:17:36.562381 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:36.562327 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vnw7h" podStartSLOduration=1.3599456939999999 podStartE2EDuration="1.562307139s" totalImagesPullingTime="202.361445ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:17:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:17:36.561873122 +0000 UTC m=+752.959531501" watchObservedRunningTime="2026-07-13 17:17:36.562307139 +0000 UTC m=+752.959965518" Jul 13 17:17:40.739960 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:40.739917 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-49fj9" Jul 13 17:17:40.740346 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:40.739979 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-49fj9" Jul 13 17:17:40.751871 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:40.751847 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-49fj9" Jul 13 17:17:41.577578 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:41.577534 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-49fj9" Jul 13 17:17:45.430067 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:45.430034 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vnw7h" Jul 13 17:17:45.430437 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:45.430082 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vnw7h" Jul 13 17:17:45.442812 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:45.442783 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vnw7h" Jul 13 17:17:45.598304 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:17:45.598278 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vnw7h" Jul 13 17:20:04.091946 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:20:04.091865 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-hzcql_c40e6e88-ab3f-4e3c-a0e7-2400bda333eb/console-operator/1.log" Jul 13 17:20:04.095202 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:20:04.095179 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-hzcql_c40e6e88-ab3f-4e3c-a0e7-2400bda333eb/console-operator/1.log" Jul 13 17:20:04.099973 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:20:04.099956 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q2pnb_6a531b70-4a58-4752-8fa8-5ba634b542ea/ovn-acl-logging/0.log" Jul 13 17:20:04.103576 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:20:04.103544 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q2pnb_6a531b70-4a58-4752-8fa8-5ba634b542ea/ovn-acl-logging/0.log" Jul 13 17:20:20.702096 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:20:20.702061 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-db977844c-9jf6c"] Jul 13 17:20:20.702490 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:20:20.702256 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-db977844c-9jf6c" podUID="12352f14-702e-45f7-81d8-7cb31d934475" containerName="manager" containerID="cri-o://61bd65545f79966e5809c3a5b18ca1f7ac344eb0bdaaa0cdfdda38af4241af03" gracePeriod=30 Jul 13 17:20:20.957111 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:20:20.957058 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-db977844c-9jf6c" Jul 13 17:20:21.106754 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:20:21.106721 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/12352f14-702e-45f7-81d8-7cb31d934475-webhook-cert\") pod \"12352f14-702e-45f7-81d8-7cb31d934475\" (UID: \"12352f14-702e-45f7-81d8-7cb31d934475\") " Jul 13 17:20:21.106921 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:20:21.106808 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-92mcv\" (UniqueName: \"kubernetes.io/projected/12352f14-702e-45f7-81d8-7cb31d934475-kube-api-access-92mcv\") pod \"12352f14-702e-45f7-81d8-7cb31d934475\" (UID: \"12352f14-702e-45f7-81d8-7cb31d934475\") " Jul 13 17:20:21.108864 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:20:21.108831 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/12352f14-702e-45f7-81d8-7cb31d934475-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "12352f14-702e-45f7-81d8-7cb31d934475" (UID: "12352f14-702e-45f7-81d8-7cb31d934475"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 17:20:21.108980 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:20:21.108879 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12352f14-702e-45f7-81d8-7cb31d934475-kube-api-access-92mcv" (OuterVolumeSpecName: "kube-api-access-92mcv") pod "12352f14-702e-45f7-81d8-7cb31d934475" (UID: "12352f14-702e-45f7-81d8-7cb31d934475"). InnerVolumeSpecName "kube-api-access-92mcv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:20:21.207531 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:20:21.207467 2569 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/12352f14-702e-45f7-81d8-7cb31d934475-webhook-cert\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:20:21.207531 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:20:21.207500 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-92mcv\" (UniqueName: \"kubernetes.io/projected/12352f14-702e-45f7-81d8-7cb31d934475-kube-api-access-92mcv\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:20:21.301373 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:20:21.301348 2569 generic.go:356] "Generic (PLEG): container finished" podID="12352f14-702e-45f7-81d8-7cb31d934475" containerID="61bd65545f79966e5809c3a5b18ca1f7ac344eb0bdaaa0cdfdda38af4241af03" exitCode=0 Jul 13 17:20:21.301513 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:20:21.301399 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-db977844c-9jf6c" Jul 13 17:20:21.301513 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:20:21.301431 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-db977844c-9jf6c" event={"ID":"12352f14-702e-45f7-81d8-7cb31d934475","Type":"ContainerDied","Data":"61bd65545f79966e5809c3a5b18ca1f7ac344eb0bdaaa0cdfdda38af4241af03"} Jul 13 17:20:21.301513 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:20:21.301487 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-db977844c-9jf6c" event={"ID":"12352f14-702e-45f7-81d8-7cb31d934475","Type":"ContainerDied","Data":"ab0a27c4b8a85149cedc853285a5fb42a3b271e7c09c1f8942cf3d87809ab1a7"} Jul 13 17:20:21.301513 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:20:21.301512 2569 scope.go:117] "RemoveContainer" containerID="61bd65545f79966e5809c3a5b18ca1f7ac344eb0bdaaa0cdfdda38af4241af03" Jul 13 17:20:21.317707 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:20:21.317687 2569 scope.go:117] "RemoveContainer" containerID="61bd65545f79966e5809c3a5b18ca1f7ac344eb0bdaaa0cdfdda38af4241af03" Jul 13 17:20:21.317968 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:20:21.317950 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61bd65545f79966e5809c3a5b18ca1f7ac344eb0bdaaa0cdfdda38af4241af03\": container with ID starting with 61bd65545f79966e5809c3a5b18ca1f7ac344eb0bdaaa0cdfdda38af4241af03 not found: ID does not exist" containerID="61bd65545f79966e5809c3a5b18ca1f7ac344eb0bdaaa0cdfdda38af4241af03" Jul 13 17:20:21.318019 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:20:21.317977 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61bd65545f79966e5809c3a5b18ca1f7ac344eb0bdaaa0cdfdda38af4241af03"} err="failed to get container status \"61bd65545f79966e5809c3a5b18ca1f7ac344eb0bdaaa0cdfdda38af4241af03\": rpc error: code = NotFound desc = could not find container \"61bd65545f79966e5809c3a5b18ca1f7ac344eb0bdaaa0cdfdda38af4241af03\": container with ID starting with 61bd65545f79966e5809c3a5b18ca1f7ac344eb0bdaaa0cdfdda38af4241af03 not found: ID does not exist" Jul 13 17:20:21.326498 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:20:21.326476 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-db977844c-9jf6c"] Jul 13 17:20:21.332074 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:20:21.332052 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-db977844c-9jf6c"] Jul 13 17:20:22.136812 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:20:22.136778 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12352f14-702e-45f7-81d8-7cb31d934475" path="/var/lib/kubelet/pods/12352f14-702e-45f7-81d8-7cb31d934475/volumes" Jul 13 17:20:22.163270 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:20:22.163243 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-db977844c-wvpjx"] Jul 13 17:20:22.163601 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:20:22.163589 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12352f14-702e-45f7-81d8-7cb31d934475" containerName="manager" Jul 13 17:20:22.163654 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:20:22.163603 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="12352f14-702e-45f7-81d8-7cb31d934475" containerName="manager" Jul 13 17:20:22.163689 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:20:22.163676 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="12352f14-702e-45f7-81d8-7cb31d934475" containerName="manager" Jul 13 17:20:22.167688 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:20:22.167672 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-db977844c-wvpjx" Jul 13 17:20:22.169695 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:20:22.169675 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 13 17:20:22.170231 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:20:22.170215 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 13 17:20:22.170314 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:20:22.170263 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-7hpr6\"" Jul 13 17:20:22.176321 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:20:22.176300 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-db977844c-wvpjx"] Jul 13 17:20:22.318918 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:20:22.318888 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cf5sm\" (UniqueName: \"kubernetes.io/projected/2dd1f384-9047-4f6f-9865-4138986a2532-kube-api-access-cf5sm\") pod \"maas-controller-db977844c-wvpjx\" (UID: \"2dd1f384-9047-4f6f-9865-4138986a2532\") " pod="opendatahub/maas-controller-db977844c-wvpjx" Jul 13 17:20:22.319082 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:20:22.318928 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2dd1f384-9047-4f6f-9865-4138986a2532-webhook-cert\") pod \"maas-controller-db977844c-wvpjx\" (UID: \"2dd1f384-9047-4f6f-9865-4138986a2532\") " pod="opendatahub/maas-controller-db977844c-wvpjx" Jul 13 17:20:22.420154 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:20:22.420072 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2dd1f384-9047-4f6f-9865-4138986a2532-webhook-cert\") pod \"maas-controller-db977844c-wvpjx\" (UID: \"2dd1f384-9047-4f6f-9865-4138986a2532\") " pod="opendatahub/maas-controller-db977844c-wvpjx" Jul 13 17:20:22.420287 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:20:22.420166 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cf5sm\" (UniqueName: \"kubernetes.io/projected/2dd1f384-9047-4f6f-9865-4138986a2532-kube-api-access-cf5sm\") pod \"maas-controller-db977844c-wvpjx\" (UID: \"2dd1f384-9047-4f6f-9865-4138986a2532\") " pod="opendatahub/maas-controller-db977844c-wvpjx" Jul 13 17:20:22.422726 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:20:22.422699 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2dd1f384-9047-4f6f-9865-4138986a2532-webhook-cert\") pod \"maas-controller-db977844c-wvpjx\" (UID: \"2dd1f384-9047-4f6f-9865-4138986a2532\") " pod="opendatahub/maas-controller-db977844c-wvpjx" Jul 13 17:20:22.429445 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:20:22.429419 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cf5sm\" (UniqueName: \"kubernetes.io/projected/2dd1f384-9047-4f6f-9865-4138986a2532-kube-api-access-cf5sm\") pod \"maas-controller-db977844c-wvpjx\" (UID: \"2dd1f384-9047-4f6f-9865-4138986a2532\") " pod="opendatahub/maas-controller-db977844c-wvpjx" Jul 13 17:20:22.479009 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:20:22.478967 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-db977844c-wvpjx" Jul 13 17:20:22.823395 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:20:22.823364 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-db977844c-wvpjx"] Jul 13 17:20:22.825246 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:20:22.825217 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2dd1f384_9047_4f6f_9865_4138986a2532.slice/crio-7d9901e15b62aecf3dd53fd32c0efb0b4fbf72a7745c0e637c4d4f0151e2b23f WatchSource:0}: Error finding container 7d9901e15b62aecf3dd53fd32c0efb0b4fbf72a7745c0e637c4d4f0151e2b23f: Status 404 returned error can't find the container with id 7d9901e15b62aecf3dd53fd32c0efb0b4fbf72a7745c0e637c4d4f0151e2b23f Jul 13 17:20:23.313112 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:20:23.313078 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-db977844c-wvpjx" event={"ID":"2dd1f384-9047-4f6f-9865-4138986a2532","Type":"ContainerStarted","Data":"7d9901e15b62aecf3dd53fd32c0efb0b4fbf72a7745c0e637c4d4f0151e2b23f"} Jul 13 17:20:24.319209 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:20:24.319171 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-db977844c-wvpjx" event={"ID":"2dd1f384-9047-4f6f-9865-4138986a2532","Type":"ContainerStarted","Data":"6275b6b2fb0f7da7de9e920aae89957a2c81ce3125319a4e423ef8efb55e39cd"} Jul 13 17:20:24.319613 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:20:24.319326 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-db977844c-wvpjx" Jul 13 17:20:24.336222 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:20:24.336177 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-db977844c-wvpjx" podStartSLOduration=1.818574423 podStartE2EDuration="2.336163196s" totalImagesPullingTime="517.588773ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:20:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:20:24.334979736 +0000 UTC m=+920.732638128" watchObservedRunningTime="2026-07-13 17:20:24.336163196 +0000 UTC m=+920.733821575" Jul 13 17:20:35.325252 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:20:35.325220 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-db977844c-wvpjx" Jul 13 17:25:04.121940 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:25:04.121908 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-hzcql_c40e6e88-ab3f-4e3c-a0e7-2400bda333eb/console-operator/1.log" Jul 13 17:25:04.122882 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:25:04.122851 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-hzcql_c40e6e88-ab3f-4e3c-a0e7-2400bda333eb/console-operator/1.log" Jul 13 17:25:04.130556 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:25:04.130536 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q2pnb_6a531b70-4a58-4752-8fa8-5ba634b542ea/ovn-acl-logging/0.log" Jul 13 17:25:04.131414 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:25:04.131394 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q2pnb_6a531b70-4a58-4752-8fa8-5ba634b542ea/ovn-acl-logging/0.log" Jul 13 17:30:04.152524 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:30:04.152494 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-hzcql_c40e6e88-ab3f-4e3c-a0e7-2400bda333eb/console-operator/1.log" Jul 13 17:30:04.153780 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:30:04.153759 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-hzcql_c40e6e88-ab3f-4e3c-a0e7-2400bda333eb/console-operator/1.log" Jul 13 17:30:04.162141 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:30:04.162121 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q2pnb_6a531b70-4a58-4752-8fa8-5ba634b542ea/ovn-acl-logging/0.log" Jul 13 17:30:04.163368 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:30:04.163347 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q2pnb_6a531b70-4a58-4752-8fa8-5ba634b542ea/ovn-acl-logging/0.log" Jul 13 17:35:04.182896 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:35:04.182866 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-hzcql_c40e6e88-ab3f-4e3c-a0e7-2400bda333eb/console-operator/1.log" Jul 13 17:35:04.185190 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:35:04.185169 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-hzcql_c40e6e88-ab3f-4e3c-a0e7-2400bda333eb/console-operator/1.log" Jul 13 17:35:04.192412 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:35:04.192393 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q2pnb_6a531b70-4a58-4752-8fa8-5ba634b542ea/ovn-acl-logging/0.log" Jul 13 17:35:04.194101 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:35:04.194085 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q2pnb_6a531b70-4a58-4752-8fa8-5ba634b542ea/ovn-acl-logging/0.log" Jul 13 17:39:29.024201 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:39:29.024118 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-f4b51184/e2e-lifecycle-model-f4b51184-kserve-6fbd55f67-l5xwf"] Jul 13 17:39:29.027847 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:39:29.027825 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-f4b51184/e2e-lifecycle-model-f4b51184-kserve-6fbd55f67-l5xwf" Jul 13 17:39:29.030571 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:39:29.030543 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-f4b51184\"/\"default-dockercfg-78z5s\"" Jul 13 17:39:29.030714 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:39:29.030582 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-f4b51184\"/\"e2e-lifecycle-model-f4b51184-kserve-self-signed-certs\"" Jul 13 17:39:29.030714 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:39:29.030585 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-f4b51184\"/\"kube-root-ca.crt\"" Jul 13 17:39:29.030714 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:39:29.030586 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-f4b51184\"/\"openshift-service-ca.crt\"" Jul 13 17:39:29.038976 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:39:29.038954 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-f4b51184/e2e-lifecycle-model-f4b51184-kserve-6fbd55f67-l5xwf"] Jul 13 17:39:29.077238 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:39:29.077217 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6db9929-8d61-49f8-b9d9-f289579bc3cf-tls-certs\") pod \"e2e-lifecycle-model-f4b51184-kserve-6fbd55f67-l5xwf\" (UID: \"f6db9929-8d61-49f8-b9d9-f289579bc3cf\") " pod="ai-tenant-e2e-mt-f4b51184/e2e-lifecycle-model-f4b51184-kserve-6fbd55f67-l5xwf" Jul 13 17:39:29.077346 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:39:29.077250 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6db9929-8d61-49f8-b9d9-f289579bc3cf-dshm\") pod \"e2e-lifecycle-model-f4b51184-kserve-6fbd55f67-l5xwf\" (UID: \"f6db9929-8d61-49f8-b9d9-f289579bc3cf\") " pod="ai-tenant-e2e-mt-f4b51184/e2e-lifecycle-model-f4b51184-kserve-6fbd55f67-l5xwf" Jul 13 17:39:29.077346 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:39:29.077280 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6db9929-8d61-49f8-b9d9-f289579bc3cf-model-cache\") pod \"e2e-lifecycle-model-f4b51184-kserve-6fbd55f67-l5xwf\" (UID: \"f6db9929-8d61-49f8-b9d9-f289579bc3cf\") " pod="ai-tenant-e2e-mt-f4b51184/e2e-lifecycle-model-f4b51184-kserve-6fbd55f67-l5xwf" Jul 13 17:39:29.077471 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:39:29.077404 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6db9929-8d61-49f8-b9d9-f289579bc3cf-tmp-dir\") pod \"e2e-lifecycle-model-f4b51184-kserve-6fbd55f67-l5xwf\" (UID: \"f6db9929-8d61-49f8-b9d9-f289579bc3cf\") " pod="ai-tenant-e2e-mt-f4b51184/e2e-lifecycle-model-f4b51184-kserve-6fbd55f67-l5xwf" Jul 13 17:39:29.077528 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:39:29.077465 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5szb\" (UniqueName: \"kubernetes.io/projected/f6db9929-8d61-49f8-b9d9-f289579bc3cf-kube-api-access-d5szb\") pod \"e2e-lifecycle-model-f4b51184-kserve-6fbd55f67-l5xwf\" (UID: \"f6db9929-8d61-49f8-b9d9-f289579bc3cf\") " pod="ai-tenant-e2e-mt-f4b51184/e2e-lifecycle-model-f4b51184-kserve-6fbd55f67-l5xwf" Jul 13 17:39:29.077528 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:39:29.077497 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6db9929-8d61-49f8-b9d9-f289579bc3cf-home\") pod \"e2e-lifecycle-model-f4b51184-kserve-6fbd55f67-l5xwf\" (UID: \"f6db9929-8d61-49f8-b9d9-f289579bc3cf\") " pod="ai-tenant-e2e-mt-f4b51184/e2e-lifecycle-model-f4b51184-kserve-6fbd55f67-l5xwf" Jul 13 17:39:29.178718 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:39:29.178685 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5szb\" (UniqueName: \"kubernetes.io/projected/f6db9929-8d61-49f8-b9d9-f289579bc3cf-kube-api-access-d5szb\") pod \"e2e-lifecycle-model-f4b51184-kserve-6fbd55f67-l5xwf\" (UID: \"f6db9929-8d61-49f8-b9d9-f289579bc3cf\") " pod="ai-tenant-e2e-mt-f4b51184/e2e-lifecycle-model-f4b51184-kserve-6fbd55f67-l5xwf" Jul 13 17:39:29.178718 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:39:29.178720 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6db9929-8d61-49f8-b9d9-f289579bc3cf-home\") pod \"e2e-lifecycle-model-f4b51184-kserve-6fbd55f67-l5xwf\" (UID: \"f6db9929-8d61-49f8-b9d9-f289579bc3cf\") " pod="ai-tenant-e2e-mt-f4b51184/e2e-lifecycle-model-f4b51184-kserve-6fbd55f67-l5xwf" Jul 13 17:39:29.178939 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:39:29.178741 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6db9929-8d61-49f8-b9d9-f289579bc3cf-tls-certs\") pod \"e2e-lifecycle-model-f4b51184-kserve-6fbd55f67-l5xwf\" (UID: \"f6db9929-8d61-49f8-b9d9-f289579bc3cf\") " pod="ai-tenant-e2e-mt-f4b51184/e2e-lifecycle-model-f4b51184-kserve-6fbd55f67-l5xwf" Jul 13 17:39:29.178939 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:39:29.178771 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6db9929-8d61-49f8-b9d9-f289579bc3cf-dshm\") pod \"e2e-lifecycle-model-f4b51184-kserve-6fbd55f67-l5xwf\" (UID: \"f6db9929-8d61-49f8-b9d9-f289579bc3cf\") " pod="ai-tenant-e2e-mt-f4b51184/e2e-lifecycle-model-f4b51184-kserve-6fbd55f67-l5xwf" Jul 13 17:39:29.178939 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:39:29.178890 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6db9929-8d61-49f8-b9d9-f289579bc3cf-model-cache\") pod \"e2e-lifecycle-model-f4b51184-kserve-6fbd55f67-l5xwf\" (UID: \"f6db9929-8d61-49f8-b9d9-f289579bc3cf\") " pod="ai-tenant-e2e-mt-f4b51184/e2e-lifecycle-model-f4b51184-kserve-6fbd55f67-l5xwf" Jul 13 17:39:29.179091 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:39:29.178981 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6db9929-8d61-49f8-b9d9-f289579bc3cf-tmp-dir\") pod \"e2e-lifecycle-model-f4b51184-kserve-6fbd55f67-l5xwf\" (UID: \"f6db9929-8d61-49f8-b9d9-f289579bc3cf\") " pod="ai-tenant-e2e-mt-f4b51184/e2e-lifecycle-model-f4b51184-kserve-6fbd55f67-l5xwf" Jul 13 17:39:29.179321 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:39:29.179291 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6db9929-8d61-49f8-b9d9-f289579bc3cf-model-cache\") pod \"e2e-lifecycle-model-f4b51184-kserve-6fbd55f67-l5xwf\" (UID: \"f6db9929-8d61-49f8-b9d9-f289579bc3cf\") " pod="ai-tenant-e2e-mt-f4b51184/e2e-lifecycle-model-f4b51184-kserve-6fbd55f67-l5xwf" Jul 13 17:39:29.179321 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:39:29.179305 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6db9929-8d61-49f8-b9d9-f289579bc3cf-tmp-dir\") pod \"e2e-lifecycle-model-f4b51184-kserve-6fbd55f67-l5xwf\" (UID: \"f6db9929-8d61-49f8-b9d9-f289579bc3cf\") " pod="ai-tenant-e2e-mt-f4b51184/e2e-lifecycle-model-f4b51184-kserve-6fbd55f67-l5xwf" Jul 13 17:39:29.179459 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:39:29.179349 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6db9929-8d61-49f8-b9d9-f289579bc3cf-home\") pod \"e2e-lifecycle-model-f4b51184-kserve-6fbd55f67-l5xwf\" (UID: \"f6db9929-8d61-49f8-b9d9-f289579bc3cf\") " pod="ai-tenant-e2e-mt-f4b51184/e2e-lifecycle-model-f4b51184-kserve-6fbd55f67-l5xwf" Jul 13 17:39:29.181015 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:39:29.180992 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6db9929-8d61-49f8-b9d9-f289579bc3cf-dshm\") pod \"e2e-lifecycle-model-f4b51184-kserve-6fbd55f67-l5xwf\" (UID: \"f6db9929-8d61-49f8-b9d9-f289579bc3cf\") " pod="ai-tenant-e2e-mt-f4b51184/e2e-lifecycle-model-f4b51184-kserve-6fbd55f67-l5xwf" Jul 13 17:39:29.181176 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:39:29.181157 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6db9929-8d61-49f8-b9d9-f289579bc3cf-tls-certs\") pod \"e2e-lifecycle-model-f4b51184-kserve-6fbd55f67-l5xwf\" (UID: \"f6db9929-8d61-49f8-b9d9-f289579bc3cf\") " pod="ai-tenant-e2e-mt-f4b51184/e2e-lifecycle-model-f4b51184-kserve-6fbd55f67-l5xwf" Jul 13 17:39:29.187751 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:39:29.187730 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5szb\" (UniqueName: \"kubernetes.io/projected/f6db9929-8d61-49f8-b9d9-f289579bc3cf-kube-api-access-d5szb\") pod \"e2e-lifecycle-model-f4b51184-kserve-6fbd55f67-l5xwf\" (UID: \"f6db9929-8d61-49f8-b9d9-f289579bc3cf\") " pod="ai-tenant-e2e-mt-f4b51184/e2e-lifecycle-model-f4b51184-kserve-6fbd55f67-l5xwf" Jul 13 17:39:29.337311 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:39:29.337285 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-f4b51184/e2e-lifecycle-model-f4b51184-kserve-6fbd55f67-l5xwf" Jul 13 17:39:29.469311 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:39:29.469286 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-f4b51184/e2e-lifecycle-model-f4b51184-kserve-6fbd55f67-l5xwf"] Jul 13 17:39:29.472073 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:39:29.472042 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf6db9929_8d61_49f8_b9d9_f289579bc3cf.slice/crio-cc891244d1d79b390b82678bc00331717139473c8e03120f8eb5a001b5b659d1 WatchSource:0}: Error finding container cc891244d1d79b390b82678bc00331717139473c8e03120f8eb5a001b5b659d1: Status 404 returned error can't find the container with id cc891244d1d79b390b82678bc00331717139473c8e03120f8eb5a001b5b659d1 Jul 13 17:39:29.631093 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:39:29.631013 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-f4b51184/e2e-lifecycle-model-f4b51184-kserve-6fbd55f67-l5xwf" event={"ID":"f6db9929-8d61-49f8-b9d9-f289579bc3cf","Type":"ContainerStarted","Data":"08d3929844cade3174d786c7bb50bb6d74f847df890b3caa9d7f0640ec151fe7"} Jul 13 17:39:29.631093 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:39:29.631066 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-f4b51184/e2e-lifecycle-model-f4b51184-kserve-6fbd55f67-l5xwf" event={"ID":"f6db9929-8d61-49f8-b9d9-f289579bc3cf","Type":"ContainerStarted","Data":"cc891244d1d79b390b82678bc00331717139473c8e03120f8eb5a001b5b659d1"} Jul 13 17:39:29.651350 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:39:29.651305 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-f4b51184/e2e-lifecycle-model-f4b51184-kserve-6fbd55f67-l5xwf" podStartSLOduration=0.651290701 podStartE2EDuration="651.290701ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:39:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:39:29.650738794 +0000 UTC m=+2066.048397173" watchObservedRunningTime="2026-07-13 17:39:29.651290701 +0000 UTC m=+2066.048949080" Jul 13 17:39:39.337532 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:39:39.337500 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-f4b51184/e2e-lifecycle-model-f4b51184-kserve-6fbd55f67-l5xwf" Jul 13 17:39:39.337958 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:39:39.337550 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-f4b51184/e2e-lifecycle-model-f4b51184-kserve-6fbd55f67-l5xwf" Jul 13 17:39:39.350142 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:39:39.350119 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-f4b51184/e2e-lifecycle-model-f4b51184-kserve-6fbd55f67-l5xwf" Jul 13 17:39:39.688306 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:39:39.688231 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-f4b51184/e2e-lifecycle-model-f4b51184-kserve-6fbd55f67-l5xwf" Jul 13 17:40:04.212657 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:40:04.212625 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-hzcql_c40e6e88-ab3f-4e3c-a0e7-2400bda333eb/console-operator/1.log" Jul 13 17:40:04.216353 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:40:04.216329 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-hzcql_c40e6e88-ab3f-4e3c-a0e7-2400bda333eb/console-operator/1.log" Jul 13 17:40:04.221142 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:40:04.221120 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q2pnb_6a531b70-4a58-4752-8fa8-5ba634b542ea/ovn-acl-logging/0.log" Jul 13 17:40:04.226686 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:40:04.226658 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q2pnb_6a531b70-4a58-4752-8fa8-5ba634b542ea/ovn-acl-logging/0.log" Jul 13 17:40:12.927788 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:40:12.927752 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-f4b51184/e2e-lifecycle-model-f4b51184-kserve-6fbd55f67-l5xwf"] Jul 13 17:40:27.947924 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:40:27.947867 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-f4b51184/e2e-lifecycle-model-f4b51184-kserve-6fbd55f67-l5xwf" podUID="f6db9929-8d61-49f8-b9d9-f289579bc3cf" containerName="main" containerID="cri-o://08d3929844cade3174d786c7bb50bb6d74f847df890b3caa9d7f0640ec151fe7" gracePeriod=45 Jul 13 17:40:28.200319 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:40:28.200266 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-f4b51184/e2e-lifecycle-model-f4b51184-kserve-6fbd55f67-l5xwf" Jul 13 17:40:28.275522 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:40:28.275490 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6db9929-8d61-49f8-b9d9-f289579bc3cf-dshm\") pod \"f6db9929-8d61-49f8-b9d9-f289579bc3cf\" (UID: \"f6db9929-8d61-49f8-b9d9-f289579bc3cf\") " Jul 13 17:40:28.275681 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:40:28.275539 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6db9929-8d61-49f8-b9d9-f289579bc3cf-tmp-dir\") pod \"f6db9929-8d61-49f8-b9d9-f289579bc3cf\" (UID: \"f6db9929-8d61-49f8-b9d9-f289579bc3cf\") " Jul 13 17:40:28.275681 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:40:28.275623 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d5szb\" (UniqueName: \"kubernetes.io/projected/f6db9929-8d61-49f8-b9d9-f289579bc3cf-kube-api-access-d5szb\") pod \"f6db9929-8d61-49f8-b9d9-f289579bc3cf\" (UID: \"f6db9929-8d61-49f8-b9d9-f289579bc3cf\") " Jul 13 17:40:28.275681 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:40:28.275646 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6db9929-8d61-49f8-b9d9-f289579bc3cf-tls-certs\") pod \"f6db9929-8d61-49f8-b9d9-f289579bc3cf\" (UID: \"f6db9929-8d61-49f8-b9d9-f289579bc3cf\") " Jul 13 17:40:28.275681 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:40:28.275670 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6db9929-8d61-49f8-b9d9-f289579bc3cf-model-cache\") pod \"f6db9929-8d61-49f8-b9d9-f289579bc3cf\" (UID: \"f6db9929-8d61-49f8-b9d9-f289579bc3cf\") " Jul 13 17:40:28.275896 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:40:28.275759 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6db9929-8d61-49f8-b9d9-f289579bc3cf-home\") pod \"f6db9929-8d61-49f8-b9d9-f289579bc3cf\" (UID: \"f6db9929-8d61-49f8-b9d9-f289579bc3cf\") " Jul 13 17:40:28.275896 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:40:28.275863 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6db9929-8d61-49f8-b9d9-f289579bc3cf-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f6db9929-8d61-49f8-b9d9-f289579bc3cf" (UID: "f6db9929-8d61-49f8-b9d9-f289579bc3cf"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:40:28.276079 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:40:28.276061 2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6db9929-8d61-49f8-b9d9-f289579bc3cf-tmp-dir\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:40:28.276208 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:40:28.276112 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6db9929-8d61-49f8-b9d9-f289579bc3cf-model-cache" (OuterVolumeSpecName: "model-cache") pod "f6db9929-8d61-49f8-b9d9-f289579bc3cf" (UID: "f6db9929-8d61-49f8-b9d9-f289579bc3cf"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:40:28.276282 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:40:28.276194 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6db9929-8d61-49f8-b9d9-f289579bc3cf-home" (OuterVolumeSpecName: "home") pod "f6db9929-8d61-49f8-b9d9-f289579bc3cf" (UID: "f6db9929-8d61-49f8-b9d9-f289579bc3cf"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:40:28.277805 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:40:28.277783 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6db9929-8d61-49f8-b9d9-f289579bc3cf-dshm" (OuterVolumeSpecName: "dshm") pod "f6db9929-8d61-49f8-b9d9-f289579bc3cf" (UID: "f6db9929-8d61-49f8-b9d9-f289579bc3cf"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:40:28.277917 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:40:28.277849 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6db9929-8d61-49f8-b9d9-f289579bc3cf-kube-api-access-d5szb" (OuterVolumeSpecName: "kube-api-access-d5szb") pod "f6db9929-8d61-49f8-b9d9-f289579bc3cf" (UID: "f6db9929-8d61-49f8-b9d9-f289579bc3cf"). InnerVolumeSpecName "kube-api-access-d5szb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:40:28.277917 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:40:28.277865 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6db9929-8d61-49f8-b9d9-f289579bc3cf-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f6db9929-8d61-49f8-b9d9-f289579bc3cf" (UID: "f6db9929-8d61-49f8-b9d9-f289579bc3cf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 17:40:28.377223 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:40:28.377196 2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6db9929-8d61-49f8-b9d9-f289579bc3cf-model-cache\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:40:28.377223 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:40:28.377216 2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6db9929-8d61-49f8-b9d9-f289579bc3cf-home\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:40:28.377223 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:40:28.377225 2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6db9929-8d61-49f8-b9d9-f289579bc3cf-dshm\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:40:28.377371 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:40:28.377234 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d5szb\" (UniqueName: \"kubernetes.io/projected/f6db9929-8d61-49f8-b9d9-f289579bc3cf-kube-api-access-d5szb\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:40:28.377371 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:40:28.377243 2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6db9929-8d61-49f8-b9d9-f289579bc3cf-tls-certs\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:40:28.912735 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:40:28.912704 2569 generic.go:356] "Generic (PLEG): container finished" podID="f6db9929-8d61-49f8-b9d9-f289579bc3cf" containerID="08d3929844cade3174d786c7bb50bb6d74f847df890b3caa9d7f0640ec151fe7" exitCode=0 Jul 13 17:40:28.912913 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:40:28.912748 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-f4b51184/e2e-lifecycle-model-f4b51184-kserve-6fbd55f67-l5xwf" Jul 13 17:40:28.912913 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:40:28.912784 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-f4b51184/e2e-lifecycle-model-f4b51184-kserve-6fbd55f67-l5xwf" event={"ID":"f6db9929-8d61-49f8-b9d9-f289579bc3cf","Type":"ContainerDied","Data":"08d3929844cade3174d786c7bb50bb6d74f847df890b3caa9d7f0640ec151fe7"} Jul 13 17:40:28.912913 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:40:28.912830 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-f4b51184/e2e-lifecycle-model-f4b51184-kserve-6fbd55f67-l5xwf" event={"ID":"f6db9929-8d61-49f8-b9d9-f289579bc3cf","Type":"ContainerDied","Data":"cc891244d1d79b390b82678bc00331717139473c8e03120f8eb5a001b5b659d1"} Jul 13 17:40:28.912913 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:40:28.912849 2569 scope.go:117] "RemoveContainer" containerID="08d3929844cade3174d786c7bb50bb6d74f847df890b3caa9d7f0640ec151fe7" Jul 13 17:40:28.922444 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:40:28.922426 2569 scope.go:117] "RemoveContainer" containerID="08d3929844cade3174d786c7bb50bb6d74f847df890b3caa9d7f0640ec151fe7" Jul 13 17:40:28.922689 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:40:28.922671 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08d3929844cade3174d786c7bb50bb6d74f847df890b3caa9d7f0640ec151fe7\": container with ID starting with 08d3929844cade3174d786c7bb50bb6d74f847df890b3caa9d7f0640ec151fe7 not found: ID does not exist" containerID="08d3929844cade3174d786c7bb50bb6d74f847df890b3caa9d7f0640ec151fe7" Jul 13 17:40:28.922748 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:40:28.922696 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08d3929844cade3174d786c7bb50bb6d74f847df890b3caa9d7f0640ec151fe7"} err="failed to get container status \"08d3929844cade3174d786c7bb50bb6d74f847df890b3caa9d7f0640ec151fe7\": rpc error: code = NotFound desc = could not find container \"08d3929844cade3174d786c7bb50bb6d74f847df890b3caa9d7f0640ec151fe7\": container with ID starting with 08d3929844cade3174d786c7bb50bb6d74f847df890b3caa9d7f0640ec151fe7 not found: ID does not exist" Jul 13 17:40:28.936370 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:40:28.936339 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-f4b51184/e2e-lifecycle-model-f4b51184-kserve-6fbd55f67-l5xwf"] Jul 13 17:40:28.939019 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:40:28.938997 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-f4b51184/e2e-lifecycle-model-f4b51184-kserve-6fbd55f67-l5xwf"] Jul 13 17:40:30.136858 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:40:30.136823 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6db9929-8d61-49f8-b9d9-f289579bc3cf" path="/var/lib/kubelet/pods/f6db9929-8d61-49f8-b9d9-f289579bc3cf/volumes" Jul 13 17:42:17.116051 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:17.116011 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-6a3680/test-model-6a3680-kserve-6c698946f-w6rp6"] Jul 13 17:42:17.116522 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:17.116386 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6db9929-8d61-49f8-b9d9-f289579bc3cf" containerName="main" Jul 13 17:42:17.116522 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:17.116397 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6db9929-8d61-49f8-b9d9-f289579bc3cf" containerName="main" Jul 13 17:42:17.116522 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:17.116464 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6db9929-8d61-49f8-b9d9-f289579bc3cf" containerName="main" Jul 13 17:42:17.119648 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:17.119632 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-6a3680/test-model-6a3680-kserve-6c698946f-w6rp6" Jul 13 17:42:17.122602 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:17.122580 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-6a3680\"/\"test-model-6a3680-kserve-self-signed-certs\"" Jul 13 17:42:17.122738 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:17.122587 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-6a3680\"/\"kube-root-ca.crt\"" Jul 13 17:42:17.122931 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:17.122906 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-6a3680\"/\"openshift-service-ca.crt\"" Jul 13 17:42:17.123192 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:17.123173 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-6a3680\"/\"default-dockercfg-l5pnb\"" Jul 13 17:42:17.135656 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:17.135632 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-6a3680/test-model-6a3680-kserve-6c698946f-w6rp6"] Jul 13 17:42:17.243686 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:17.243659 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbd5n\" (UniqueName: \"kubernetes.io/projected/a8f87a25-5ec0-478f-b043-08b7635e5105-kube-api-access-rbd5n\") pod \"test-model-6a3680-kserve-6c698946f-w6rp6\" (UID: \"a8f87a25-5ec0-478f-b043-08b7635e5105\") " pod="ai-tenant-e2e-inf-a-6a3680/test-model-6a3680-kserve-6c698946f-w6rp6" Jul 13 17:42:17.243792 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:17.243692 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8f87a25-5ec0-478f-b043-08b7635e5105-home\") pod \"test-model-6a3680-kserve-6c698946f-w6rp6\" (UID: \"a8f87a25-5ec0-478f-b043-08b7635e5105\") " pod="ai-tenant-e2e-inf-a-6a3680/test-model-6a3680-kserve-6c698946f-w6rp6" Jul 13 17:42:17.243792 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:17.243714 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8f87a25-5ec0-478f-b043-08b7635e5105-tmp-dir\") pod \"test-model-6a3680-kserve-6c698946f-w6rp6\" (UID: \"a8f87a25-5ec0-478f-b043-08b7635e5105\") " pod="ai-tenant-e2e-inf-a-6a3680/test-model-6a3680-kserve-6c698946f-w6rp6" Jul 13 17:42:17.243792 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:17.243738 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8f87a25-5ec0-478f-b043-08b7635e5105-dshm\") pod \"test-model-6a3680-kserve-6c698946f-w6rp6\" (UID: \"a8f87a25-5ec0-478f-b043-08b7635e5105\") " pod="ai-tenant-e2e-inf-a-6a3680/test-model-6a3680-kserve-6c698946f-w6rp6" Jul 13 17:42:17.243957 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:17.243793 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8f87a25-5ec0-478f-b043-08b7635e5105-tls-certs\") pod \"test-model-6a3680-kserve-6c698946f-w6rp6\" (UID: \"a8f87a25-5ec0-478f-b043-08b7635e5105\") " pod="ai-tenant-e2e-inf-a-6a3680/test-model-6a3680-kserve-6c698946f-w6rp6" Jul 13 17:42:17.243957 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:17.243856 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8f87a25-5ec0-478f-b043-08b7635e5105-model-cache\") pod \"test-model-6a3680-kserve-6c698946f-w6rp6\" (UID: \"a8f87a25-5ec0-478f-b043-08b7635e5105\") " pod="ai-tenant-e2e-inf-a-6a3680/test-model-6a3680-kserve-6c698946f-w6rp6" Jul 13 17:42:17.344961 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:17.344916 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbd5n\" (UniqueName: \"kubernetes.io/projected/a8f87a25-5ec0-478f-b043-08b7635e5105-kube-api-access-rbd5n\") pod \"test-model-6a3680-kserve-6c698946f-w6rp6\" (UID: \"a8f87a25-5ec0-478f-b043-08b7635e5105\") " pod="ai-tenant-e2e-inf-a-6a3680/test-model-6a3680-kserve-6c698946f-w6rp6" Jul 13 17:42:17.344961 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:17.344965 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8f87a25-5ec0-478f-b043-08b7635e5105-home\") pod \"test-model-6a3680-kserve-6c698946f-w6rp6\" (UID: \"a8f87a25-5ec0-478f-b043-08b7635e5105\") " pod="ai-tenant-e2e-inf-a-6a3680/test-model-6a3680-kserve-6c698946f-w6rp6" Jul 13 17:42:17.345202 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:17.345085 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8f87a25-5ec0-478f-b043-08b7635e5105-tmp-dir\") pod \"test-model-6a3680-kserve-6c698946f-w6rp6\" (UID: \"a8f87a25-5ec0-478f-b043-08b7635e5105\") " pod="ai-tenant-e2e-inf-a-6a3680/test-model-6a3680-kserve-6c698946f-w6rp6" Jul 13 17:42:17.345202 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:17.345138 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8f87a25-5ec0-478f-b043-08b7635e5105-dshm\") pod \"test-model-6a3680-kserve-6c698946f-w6rp6\" (UID: \"a8f87a25-5ec0-478f-b043-08b7635e5105\") " pod="ai-tenant-e2e-inf-a-6a3680/test-model-6a3680-kserve-6c698946f-w6rp6" Jul 13 17:42:17.345202 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:17.345172 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8f87a25-5ec0-478f-b043-08b7635e5105-tls-certs\") pod \"test-model-6a3680-kserve-6c698946f-w6rp6\" (UID: \"a8f87a25-5ec0-478f-b043-08b7635e5105\") " pod="ai-tenant-e2e-inf-a-6a3680/test-model-6a3680-kserve-6c698946f-w6rp6" Jul 13 17:42:17.345347 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:17.345248 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8f87a25-5ec0-478f-b043-08b7635e5105-model-cache\") pod \"test-model-6a3680-kserve-6c698946f-w6rp6\" (UID: \"a8f87a25-5ec0-478f-b043-08b7635e5105\") " pod="ai-tenant-e2e-inf-a-6a3680/test-model-6a3680-kserve-6c698946f-w6rp6" Jul 13 17:42:17.345347 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:17.345276 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8f87a25-5ec0-478f-b043-08b7635e5105-home\") pod \"test-model-6a3680-kserve-6c698946f-w6rp6\" (UID: \"a8f87a25-5ec0-478f-b043-08b7635e5105\") " pod="ai-tenant-e2e-inf-a-6a3680/test-model-6a3680-kserve-6c698946f-w6rp6" Jul 13 17:42:17.345461 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:17.345433 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8f87a25-5ec0-478f-b043-08b7635e5105-tmp-dir\") pod \"test-model-6a3680-kserve-6c698946f-w6rp6\" (UID: \"a8f87a25-5ec0-478f-b043-08b7635e5105\") " pod="ai-tenant-e2e-inf-a-6a3680/test-model-6a3680-kserve-6c698946f-w6rp6" Jul 13 17:42:17.345648 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:17.345625 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8f87a25-5ec0-478f-b043-08b7635e5105-model-cache\") pod \"test-model-6a3680-kserve-6c698946f-w6rp6\" (UID: \"a8f87a25-5ec0-478f-b043-08b7635e5105\") " pod="ai-tenant-e2e-inf-a-6a3680/test-model-6a3680-kserve-6c698946f-w6rp6" Jul 13 17:42:17.347349 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:17.347320 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8f87a25-5ec0-478f-b043-08b7635e5105-dshm\") pod \"test-model-6a3680-kserve-6c698946f-w6rp6\" (UID: \"a8f87a25-5ec0-478f-b043-08b7635e5105\") " pod="ai-tenant-e2e-inf-a-6a3680/test-model-6a3680-kserve-6c698946f-w6rp6" Jul 13 17:42:17.347504 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:17.347485 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8f87a25-5ec0-478f-b043-08b7635e5105-tls-certs\") pod \"test-model-6a3680-kserve-6c698946f-w6rp6\" (UID: \"a8f87a25-5ec0-478f-b043-08b7635e5105\") " pod="ai-tenant-e2e-inf-a-6a3680/test-model-6a3680-kserve-6c698946f-w6rp6" Jul 13 17:42:17.353923 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:17.353898 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbd5n\" (UniqueName: \"kubernetes.io/projected/a8f87a25-5ec0-478f-b043-08b7635e5105-kube-api-access-rbd5n\") pod \"test-model-6a3680-kserve-6c698946f-w6rp6\" (UID: \"a8f87a25-5ec0-478f-b043-08b7635e5105\") " pod="ai-tenant-e2e-inf-a-6a3680/test-model-6a3680-kserve-6c698946f-w6rp6" Jul 13 17:42:17.429437 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:17.429362 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-6a3680/test-model-6a3680-kserve-6c698946f-w6rp6" Jul 13 17:42:17.572191 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:17.572155 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-6a3680/test-model-6a3680-kserve-6c698946f-w6rp6"] Jul 13 17:42:17.576712 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:42:17.576683 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda8f87a25_5ec0_478f_b043_08b7635e5105.slice/crio-9e8ec4b9bfffa73436735dd514778f9eda943daf607b715dce53ed3d71b0e7ea WatchSource:0}: Error finding container 9e8ec4b9bfffa73436735dd514778f9eda943daf607b715dce53ed3d71b0e7ea: Status 404 returned error can't find the container with id 9e8ec4b9bfffa73436735dd514778f9eda943daf607b715dce53ed3d71b0e7ea Jul 13 17:42:18.415072 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:18.415036 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-6a3680/test-model-6a3680-kserve-6c698946f-w6rp6" event={"ID":"a8f87a25-5ec0-478f-b043-08b7635e5105","Type":"ContainerStarted","Data":"7a58ff648a5bafbe49884196d8a71da1f9f3de38c13f49077a55f81423767a3d"} Jul 13 17:42:18.415072 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:18.415075 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-6a3680/test-model-6a3680-kserve-6c698946f-w6rp6" event={"ID":"a8f87a25-5ec0-478f-b043-08b7635e5105","Type":"ContainerStarted","Data":"9e8ec4b9bfffa73436735dd514778f9eda943daf607b715dce53ed3d71b0e7ea"} Jul 13 17:42:18.441102 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:18.441055 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-6a3680/test-model-6a3680-kserve-6c698946f-w6rp6" podStartSLOduration=1.441039814 podStartE2EDuration="1.441039814s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:42:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:42:18.439732483 +0000 UTC m=+2234.837390889" watchObservedRunningTime="2026-07-13 17:42:18.441039814 +0000 UTC m=+2234.838698193" Jul 13 17:42:27.429696 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:27.429598 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-6a3680/test-model-6a3680-kserve-6c698946f-w6rp6" Jul 13 17:42:27.429696 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:27.429656 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-6a3680/test-model-6a3680-kserve-6c698946f-w6rp6" Jul 13 17:42:27.442030 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:27.442004 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-6a3680/test-model-6a3680-kserve-6c698946f-w6rp6" Jul 13 17:42:27.469270 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:27.469240 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-6a3680/test-model-6a3680-kserve-6c698946f-w6rp6" Jul 13 17:42:39.146462 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:39.146431 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-587790/test-model-587790-kserve-7d5f474545-zrqgl"] Jul 13 17:42:39.150064 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:39.150047 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-587790/test-model-587790-kserve-7d5f474545-zrqgl" Jul 13 17:42:39.153355 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:39.153333 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-587790\"/\"kube-root-ca.crt\"" Jul 13 17:42:39.154112 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:39.154087 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-587790\"/\"default-dockercfg-87gk4\"" Jul 13 17:42:39.154230 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:39.154087 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-587790\"/\"openshift-service-ca.crt\"" Jul 13 17:42:39.154230 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:39.154170 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-587790\"/\"test-model-587790-kserve-self-signed-certs\"" Jul 13 17:42:39.159973 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:39.159949 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-587790/test-model-587790-kserve-7d5f474545-zrqgl"] Jul 13 17:42:39.224912 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:39.224884 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c79607c-1d48-4bea-b676-6f23cf5ad130-dshm\") pod \"test-model-587790-kserve-7d5f474545-zrqgl\" (UID: \"4c79607c-1d48-4bea-b676-6f23cf5ad130\") " pod="ai-tenant-e2e-inf-b-587790/test-model-587790-kserve-7d5f474545-zrqgl" Jul 13 17:42:39.224912 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:39.224913 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c79607c-1d48-4bea-b676-6f23cf5ad130-tmp-dir\") pod \"test-model-587790-kserve-7d5f474545-zrqgl\" (UID: \"4c79607c-1d48-4bea-b676-6f23cf5ad130\") " pod="ai-tenant-e2e-inf-b-587790/test-model-587790-kserve-7d5f474545-zrqgl" Jul 13 17:42:39.225064 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:39.224949 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dkbw\" (UniqueName: \"kubernetes.io/projected/4c79607c-1d48-4bea-b676-6f23cf5ad130-kube-api-access-5dkbw\") pod \"test-model-587790-kserve-7d5f474545-zrqgl\" (UID: \"4c79607c-1d48-4bea-b676-6f23cf5ad130\") " pod="ai-tenant-e2e-inf-b-587790/test-model-587790-kserve-7d5f474545-zrqgl" Jul 13 17:42:39.225064 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:39.225021 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c79607c-1d48-4bea-b676-6f23cf5ad130-tls-certs\") pod \"test-model-587790-kserve-7d5f474545-zrqgl\" (UID: \"4c79607c-1d48-4bea-b676-6f23cf5ad130\") " pod="ai-tenant-e2e-inf-b-587790/test-model-587790-kserve-7d5f474545-zrqgl" Jul 13 17:42:39.225064 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:39.225047 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c79607c-1d48-4bea-b676-6f23cf5ad130-home\") pod \"test-model-587790-kserve-7d5f474545-zrqgl\" (UID: \"4c79607c-1d48-4bea-b676-6f23cf5ad130\") " pod="ai-tenant-e2e-inf-b-587790/test-model-587790-kserve-7d5f474545-zrqgl" Jul 13 17:42:39.225206 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:39.225089 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c79607c-1d48-4bea-b676-6f23cf5ad130-model-cache\") pod \"test-model-587790-kserve-7d5f474545-zrqgl\" (UID: \"4c79607c-1d48-4bea-b676-6f23cf5ad130\") " pod="ai-tenant-e2e-inf-b-587790/test-model-587790-kserve-7d5f474545-zrqgl" Jul 13 17:42:39.325671 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:39.325637 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c79607c-1d48-4bea-b676-6f23cf5ad130-model-cache\") pod \"test-model-587790-kserve-7d5f474545-zrqgl\" (UID: \"4c79607c-1d48-4bea-b676-6f23cf5ad130\") " pod="ai-tenant-e2e-inf-b-587790/test-model-587790-kserve-7d5f474545-zrqgl" Jul 13 17:42:39.325808 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:39.325696 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c79607c-1d48-4bea-b676-6f23cf5ad130-dshm\") pod \"test-model-587790-kserve-7d5f474545-zrqgl\" (UID: \"4c79607c-1d48-4bea-b676-6f23cf5ad130\") " pod="ai-tenant-e2e-inf-b-587790/test-model-587790-kserve-7d5f474545-zrqgl" Jul 13 17:42:39.325876 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:39.325804 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c79607c-1d48-4bea-b676-6f23cf5ad130-tmp-dir\") pod \"test-model-587790-kserve-7d5f474545-zrqgl\" (UID: \"4c79607c-1d48-4bea-b676-6f23cf5ad130\") " pod="ai-tenant-e2e-inf-b-587790/test-model-587790-kserve-7d5f474545-zrqgl" Jul 13 17:42:39.325930 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:39.325875 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dkbw\" (UniqueName: \"kubernetes.io/projected/4c79607c-1d48-4bea-b676-6f23cf5ad130-kube-api-access-5dkbw\") pod \"test-model-587790-kserve-7d5f474545-zrqgl\" (UID: \"4c79607c-1d48-4bea-b676-6f23cf5ad130\") " pod="ai-tenant-e2e-inf-b-587790/test-model-587790-kserve-7d5f474545-zrqgl" Jul 13 17:42:39.325986 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:39.325927 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c79607c-1d48-4bea-b676-6f23cf5ad130-tls-certs\") pod \"test-model-587790-kserve-7d5f474545-zrqgl\" (UID: \"4c79607c-1d48-4bea-b676-6f23cf5ad130\") " pod="ai-tenant-e2e-inf-b-587790/test-model-587790-kserve-7d5f474545-zrqgl" Jul 13 17:42:39.325986 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:39.325959 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c79607c-1d48-4bea-b676-6f23cf5ad130-home\") pod \"test-model-587790-kserve-7d5f474545-zrqgl\" (UID: \"4c79607c-1d48-4bea-b676-6f23cf5ad130\") " pod="ai-tenant-e2e-inf-b-587790/test-model-587790-kserve-7d5f474545-zrqgl" Jul 13 17:42:39.326090 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:39.325997 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c79607c-1d48-4bea-b676-6f23cf5ad130-model-cache\") pod \"test-model-587790-kserve-7d5f474545-zrqgl\" (UID: \"4c79607c-1d48-4bea-b676-6f23cf5ad130\") " pod="ai-tenant-e2e-inf-b-587790/test-model-587790-kserve-7d5f474545-zrqgl" Jul 13 17:42:39.326143 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:39.326117 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c79607c-1d48-4bea-b676-6f23cf5ad130-tmp-dir\") pod \"test-model-587790-kserve-7d5f474545-zrqgl\" (UID: \"4c79607c-1d48-4bea-b676-6f23cf5ad130\") " pod="ai-tenant-e2e-inf-b-587790/test-model-587790-kserve-7d5f474545-zrqgl" Jul 13 17:42:39.326315 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:39.326296 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c79607c-1d48-4bea-b676-6f23cf5ad130-home\") pod \"test-model-587790-kserve-7d5f474545-zrqgl\" (UID: \"4c79607c-1d48-4bea-b676-6f23cf5ad130\") " pod="ai-tenant-e2e-inf-b-587790/test-model-587790-kserve-7d5f474545-zrqgl" Jul 13 17:42:39.327899 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:39.327879 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c79607c-1d48-4bea-b676-6f23cf5ad130-dshm\") pod \"test-model-587790-kserve-7d5f474545-zrqgl\" (UID: \"4c79607c-1d48-4bea-b676-6f23cf5ad130\") " pod="ai-tenant-e2e-inf-b-587790/test-model-587790-kserve-7d5f474545-zrqgl" Jul 13 17:42:39.328251 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:39.328233 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c79607c-1d48-4bea-b676-6f23cf5ad130-tls-certs\") pod \"test-model-587790-kserve-7d5f474545-zrqgl\" (UID: \"4c79607c-1d48-4bea-b676-6f23cf5ad130\") " pod="ai-tenant-e2e-inf-b-587790/test-model-587790-kserve-7d5f474545-zrqgl" Jul 13 17:42:39.334917 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:39.334896 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dkbw\" (UniqueName: \"kubernetes.io/projected/4c79607c-1d48-4bea-b676-6f23cf5ad130-kube-api-access-5dkbw\") pod \"test-model-587790-kserve-7d5f474545-zrqgl\" (UID: \"4c79607c-1d48-4bea-b676-6f23cf5ad130\") " pod="ai-tenant-e2e-inf-b-587790/test-model-587790-kserve-7d5f474545-zrqgl" Jul 13 17:42:39.461163 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:39.461093 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-587790/test-model-587790-kserve-7d5f474545-zrqgl" Jul 13 17:42:39.813024 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:39.812996 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-587790/test-model-587790-kserve-7d5f474545-zrqgl"] Jul 13 17:42:39.815750 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:42:39.815718 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4c79607c_1d48_4bea_b676_6f23cf5ad130.slice/crio-0aa9c7d57770714c567b90b7811437412c3be30989cc5f4616686fb6dee463ef WatchSource:0}: Error finding container 0aa9c7d57770714c567b90b7811437412c3be30989cc5f4616686fb6dee463ef: Status 404 returned error can't find the container with id 0aa9c7d57770714c567b90b7811437412c3be30989cc5f4616686fb6dee463ef Jul 13 17:42:40.527639 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:40.527600 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-587790/test-model-587790-kserve-7d5f474545-zrqgl" event={"ID":"4c79607c-1d48-4bea-b676-6f23cf5ad130","Type":"ContainerStarted","Data":"5562b4f9abad2e631badc30fc9ae2f3725216a12bd51605cfdb0f00701a4b65e"} Jul 13 17:42:40.527639 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:40.527642 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-587790/test-model-587790-kserve-7d5f474545-zrqgl" event={"ID":"4c79607c-1d48-4bea-b676-6f23cf5ad130","Type":"ContainerStarted","Data":"0aa9c7d57770714c567b90b7811437412c3be30989cc5f4616686fb6dee463ef"} Jul 13 17:42:40.554095 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:40.554046 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-587790/test-model-587790-kserve-7d5f474545-zrqgl" podStartSLOduration=1.554033451 podStartE2EDuration="1.554033451s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:42:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:42:40.54974112 +0000 UTC m=+2256.947399498" watchObservedRunningTime="2026-07-13 17:42:40.554033451 +0000 UTC m=+2256.951691830" Jul 13 17:42:49.461877 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:49.461841 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-587790/test-model-587790-kserve-7d5f474545-zrqgl" Jul 13 17:42:49.464264 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:49.461898 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-587790/test-model-587790-kserve-7d5f474545-zrqgl" Jul 13 17:42:49.474247 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:49.474224 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-587790/test-model-587790-kserve-7d5f474545-zrqgl" Jul 13 17:42:49.581118 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:42:49.581091 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-587790/test-model-587790-kserve-7d5f474545-zrqgl" Jul 13 17:43:10.506938 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:10.506903 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-6a3680/test-model-6a3680-kserve-6c698946f-w6rp6"] Jul 13 17:43:11.457289 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:11.457256 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-587790/test-model-587790-kserve-7d5f474545-zrqgl"] Jul 13 17:43:25.526572 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:25.526502 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-6a3680/test-model-6a3680-kserve-6c698946f-w6rp6" podUID="a8f87a25-5ec0-478f-b043-08b7635e5105" containerName="main" containerID="cri-o://7a58ff648a5bafbe49884196d8a71da1f9f3de38c13f49077a55f81423767a3d" gracePeriod=45 Jul 13 17:43:25.753533 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:25.753393 2569 generic.go:356] "Generic (PLEG): container finished" podID="a8f87a25-5ec0-478f-b043-08b7635e5105" containerID="7a58ff648a5bafbe49884196d8a71da1f9f3de38c13f49077a55f81423767a3d" exitCode=0 Jul 13 17:43:25.753533 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:25.753465 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-6a3680/test-model-6a3680-kserve-6c698946f-w6rp6" event={"ID":"a8f87a25-5ec0-478f-b043-08b7635e5105","Type":"ContainerDied","Data":"7a58ff648a5bafbe49884196d8a71da1f9f3de38c13f49077a55f81423767a3d"} Jul 13 17:43:25.773497 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:25.773476 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-6a3680/test-model-6a3680-kserve-6c698946f-w6rp6" Jul 13 17:43:25.914528 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:25.914497 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8f87a25-5ec0-478f-b043-08b7635e5105-tmp-dir\") pod \"a8f87a25-5ec0-478f-b043-08b7635e5105\" (UID: \"a8f87a25-5ec0-478f-b043-08b7635e5105\") " Jul 13 17:43:25.914710 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:25.914549 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8f87a25-5ec0-478f-b043-08b7635e5105-home\") pod \"a8f87a25-5ec0-478f-b043-08b7635e5105\" (UID: \"a8f87a25-5ec0-478f-b043-08b7635e5105\") " Jul 13 17:43:25.914710 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:25.914619 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8f87a25-5ec0-478f-b043-08b7635e5105-tls-certs\") pod \"a8f87a25-5ec0-478f-b043-08b7635e5105\" (UID: \"a8f87a25-5ec0-478f-b043-08b7635e5105\") " Jul 13 17:43:25.914710 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:25.914641 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8f87a25-5ec0-478f-b043-08b7635e5105-dshm\") pod \"a8f87a25-5ec0-478f-b043-08b7635e5105\" (UID: \"a8f87a25-5ec0-478f-b043-08b7635e5105\") " Jul 13 17:43:25.914710 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:25.914686 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rbd5n\" (UniqueName: \"kubernetes.io/projected/a8f87a25-5ec0-478f-b043-08b7635e5105-kube-api-access-rbd5n\") pod \"a8f87a25-5ec0-478f-b043-08b7635e5105\" (UID: \"a8f87a25-5ec0-478f-b043-08b7635e5105\") " Jul 13 17:43:25.914921 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:25.914715 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8f87a25-5ec0-478f-b043-08b7635e5105-model-cache\") pod \"a8f87a25-5ec0-478f-b043-08b7635e5105\" (UID: \"a8f87a25-5ec0-478f-b043-08b7635e5105\") " Jul 13 17:43:25.914921 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:25.914769 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8f87a25-5ec0-478f-b043-08b7635e5105-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a8f87a25-5ec0-478f-b043-08b7635e5105" (UID: "a8f87a25-5ec0-478f-b043-08b7635e5105"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:43:25.915033 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:25.914942 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8f87a25-5ec0-478f-b043-08b7635e5105-home" (OuterVolumeSpecName: "home") pod "a8f87a25-5ec0-478f-b043-08b7635e5105" (UID: "a8f87a25-5ec0-478f-b043-08b7635e5105"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:43:25.915033 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:25.914951 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8f87a25-5ec0-478f-b043-08b7635e5105-model-cache" (OuterVolumeSpecName: "model-cache") pod "a8f87a25-5ec0-478f-b043-08b7635e5105" (UID: "a8f87a25-5ec0-478f-b043-08b7635e5105"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:43:25.915033 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:25.914962 2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8f87a25-5ec0-478f-b043-08b7635e5105-tmp-dir\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:43:25.916940 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:25.916914 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8f87a25-5ec0-478f-b043-08b7635e5105-kube-api-access-rbd5n" (OuterVolumeSpecName: "kube-api-access-rbd5n") pod "a8f87a25-5ec0-478f-b043-08b7635e5105" (UID: "a8f87a25-5ec0-478f-b043-08b7635e5105"). InnerVolumeSpecName "kube-api-access-rbd5n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:43:25.917030 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:25.916951 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8f87a25-5ec0-478f-b043-08b7635e5105-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a8f87a25-5ec0-478f-b043-08b7635e5105" (UID: "a8f87a25-5ec0-478f-b043-08b7635e5105"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 17:43:25.917030 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:25.916949 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8f87a25-5ec0-478f-b043-08b7635e5105-dshm" (OuterVolumeSpecName: "dshm") pod "a8f87a25-5ec0-478f-b043-08b7635e5105" (UID: "a8f87a25-5ec0-478f-b043-08b7635e5105"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:43:26.016099 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:26.016076 2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8f87a25-5ec0-478f-b043-08b7635e5105-dshm\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:43:26.016099 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:26.016098 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rbd5n\" (UniqueName: \"kubernetes.io/projected/a8f87a25-5ec0-478f-b043-08b7635e5105-kube-api-access-rbd5n\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:43:26.016219 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:26.016108 2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8f87a25-5ec0-478f-b043-08b7635e5105-model-cache\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:43:26.016219 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:26.016117 2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8f87a25-5ec0-478f-b043-08b7635e5105-home\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:43:26.016219 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:26.016125 2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8f87a25-5ec0-478f-b043-08b7635e5105-tls-certs\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:43:26.477060 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:26.477007 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-587790/test-model-587790-kserve-7d5f474545-zrqgl" podUID="4c79607c-1d48-4bea-b676-6f23cf5ad130" containerName="main" containerID="cri-o://5562b4f9abad2e631badc30fc9ae2f3725216a12bd51605cfdb0f00701a4b65e" gracePeriod=45 Jul 13 17:43:26.730890 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:26.730835 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-587790/test-model-587790-kserve-7d5f474545-zrqgl" Jul 13 17:43:26.759458 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:26.759436 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-6a3680/test-model-6a3680-kserve-6c698946f-w6rp6" Jul 13 17:43:26.759605 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:26.759480 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-6a3680/test-model-6a3680-kserve-6c698946f-w6rp6" event={"ID":"a8f87a25-5ec0-478f-b043-08b7635e5105","Type":"ContainerDied","Data":"9e8ec4b9bfffa73436735dd514778f9eda943daf607b715dce53ed3d71b0e7ea"} Jul 13 17:43:26.759605 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:26.759540 2569 scope.go:117] "RemoveContainer" containerID="7a58ff648a5bafbe49884196d8a71da1f9f3de38c13f49077a55f81423767a3d" Jul 13 17:43:26.760952 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:26.760930 2569 generic.go:356] "Generic (PLEG): container finished" podID="4c79607c-1d48-4bea-b676-6f23cf5ad130" containerID="5562b4f9abad2e631badc30fc9ae2f3725216a12bd51605cfdb0f00701a4b65e" exitCode=0 Jul 13 17:43:26.761030 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:26.760977 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-587790/test-model-587790-kserve-7d5f474545-zrqgl" Jul 13 17:43:26.761030 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:26.760983 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-587790/test-model-587790-kserve-7d5f474545-zrqgl" event={"ID":"4c79607c-1d48-4bea-b676-6f23cf5ad130","Type":"ContainerDied","Data":"5562b4f9abad2e631badc30fc9ae2f3725216a12bd51605cfdb0f00701a4b65e"} Jul 13 17:43:26.761030 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:26.761020 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-587790/test-model-587790-kserve-7d5f474545-zrqgl" event={"ID":"4c79607c-1d48-4bea-b676-6f23cf5ad130","Type":"ContainerDied","Data":"0aa9c7d57770714c567b90b7811437412c3be30989cc5f4616686fb6dee463ef"} Jul 13 17:43:26.768559 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:26.768516 2569 scope.go:117] "RemoveContainer" containerID="5562b4f9abad2e631badc30fc9ae2f3725216a12bd51605cfdb0f00701a4b65e" Jul 13 17:43:26.780504 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:26.778488 2569 scope.go:117] "RemoveContainer" containerID="5562b4f9abad2e631badc30fc9ae2f3725216a12bd51605cfdb0f00701a4b65e" Jul 13 17:43:26.780504 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:43:26.778836 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5562b4f9abad2e631badc30fc9ae2f3725216a12bd51605cfdb0f00701a4b65e\": container with ID starting with 5562b4f9abad2e631badc30fc9ae2f3725216a12bd51605cfdb0f00701a4b65e not found: ID does not exist" containerID="5562b4f9abad2e631badc30fc9ae2f3725216a12bd51605cfdb0f00701a4b65e" Jul 13 17:43:26.780504 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:26.778868 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5562b4f9abad2e631badc30fc9ae2f3725216a12bd51605cfdb0f00701a4b65e"} err="failed to get container status \"5562b4f9abad2e631badc30fc9ae2f3725216a12bd51605cfdb0f00701a4b65e\": rpc error: code = NotFound desc = could not find container \"5562b4f9abad2e631badc30fc9ae2f3725216a12bd51605cfdb0f00701a4b65e\": container with ID starting with 5562b4f9abad2e631badc30fc9ae2f3725216a12bd51605cfdb0f00701a4b65e not found: ID does not exist" Jul 13 17:43:26.794310 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:26.794286 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-6a3680/test-model-6a3680-kserve-6c698946f-w6rp6"] Jul 13 17:43:26.806795 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:26.806774 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-6a3680/test-model-6a3680-kserve-6c698946f-w6rp6"] Jul 13 17:43:26.822438 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:26.822417 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c79607c-1d48-4bea-b676-6f23cf5ad130-tls-certs\") pod \"4c79607c-1d48-4bea-b676-6f23cf5ad130\" (UID: \"4c79607c-1d48-4bea-b676-6f23cf5ad130\") " Jul 13 17:43:26.822495 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:26.822458 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c79607c-1d48-4bea-b676-6f23cf5ad130-tmp-dir\") pod \"4c79607c-1d48-4bea-b676-6f23cf5ad130\" (UID: \"4c79607c-1d48-4bea-b676-6f23cf5ad130\") " Jul 13 17:43:26.822495 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:26.822485 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c79607c-1d48-4bea-b676-6f23cf5ad130-home\") pod \"4c79607c-1d48-4bea-b676-6f23cf5ad130\" (UID: \"4c79607c-1d48-4bea-b676-6f23cf5ad130\") " Jul 13 17:43:26.822579 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:26.822511 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5dkbw\" (UniqueName: \"kubernetes.io/projected/4c79607c-1d48-4bea-b676-6f23cf5ad130-kube-api-access-5dkbw\") pod \"4c79607c-1d48-4bea-b676-6f23cf5ad130\" (UID: \"4c79607c-1d48-4bea-b676-6f23cf5ad130\") " Jul 13 17:43:26.822579 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:26.822547 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c79607c-1d48-4bea-b676-6f23cf5ad130-dshm\") pod \"4c79607c-1d48-4bea-b676-6f23cf5ad130\" (UID: \"4c79607c-1d48-4bea-b676-6f23cf5ad130\") " Jul 13 17:43:26.822693 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:26.822677 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c79607c-1d48-4bea-b676-6f23cf5ad130-model-cache\") pod \"4c79607c-1d48-4bea-b676-6f23cf5ad130\" (UID: \"4c79607c-1d48-4bea-b676-6f23cf5ad130\") " Jul 13 17:43:26.822748 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:26.822702 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c79607c-1d48-4bea-b676-6f23cf5ad130-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4c79607c-1d48-4bea-b676-6f23cf5ad130" (UID: "4c79607c-1d48-4bea-b676-6f23cf5ad130"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:43:26.822815 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:26.822796 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c79607c-1d48-4bea-b676-6f23cf5ad130-home" (OuterVolumeSpecName: "home") pod "4c79607c-1d48-4bea-b676-6f23cf5ad130" (UID: "4c79607c-1d48-4bea-b676-6f23cf5ad130"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:43:26.822947 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:26.822923 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c79607c-1d48-4bea-b676-6f23cf5ad130-model-cache" (OuterVolumeSpecName: "model-cache") pod "4c79607c-1d48-4bea-b676-6f23cf5ad130" (UID: "4c79607c-1d48-4bea-b676-6f23cf5ad130"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:43:26.823020 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:26.822953 2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c79607c-1d48-4bea-b676-6f23cf5ad130-tmp-dir\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:43:26.823020 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:26.822980 2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c79607c-1d48-4bea-b676-6f23cf5ad130-home\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:43:26.824578 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:26.824541 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c79607c-1d48-4bea-b676-6f23cf5ad130-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4c79607c-1d48-4bea-b676-6f23cf5ad130" (UID: "4c79607c-1d48-4bea-b676-6f23cf5ad130"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 17:43:26.824648 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:26.824623 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c79607c-1d48-4bea-b676-6f23cf5ad130-dshm" (OuterVolumeSpecName: "dshm") pod "4c79607c-1d48-4bea-b676-6f23cf5ad130" (UID: "4c79607c-1d48-4bea-b676-6f23cf5ad130"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:43:26.824760 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:26.824741 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c79607c-1d48-4bea-b676-6f23cf5ad130-kube-api-access-5dkbw" (OuterVolumeSpecName: "kube-api-access-5dkbw") pod "4c79607c-1d48-4bea-b676-6f23cf5ad130" (UID: "4c79607c-1d48-4bea-b676-6f23cf5ad130"). InnerVolumeSpecName "kube-api-access-5dkbw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:43:26.923543 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:26.923511 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5dkbw\" (UniqueName: \"kubernetes.io/projected/4c79607c-1d48-4bea-b676-6f23cf5ad130-kube-api-access-5dkbw\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:43:26.923543 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:26.923542 2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c79607c-1d48-4bea-b676-6f23cf5ad130-dshm\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:43:26.923543 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:26.923552 2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c79607c-1d48-4bea-b676-6f23cf5ad130-model-cache\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:43:26.923712 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:26.923582 2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c79607c-1d48-4bea-b676-6f23cf5ad130-tls-certs\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:43:27.082629 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:27.082602 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-587790/test-model-587790-kserve-7d5f474545-zrqgl"] Jul 13 17:43:27.086900 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:27.086876 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-587790/test-model-587790-kserve-7d5f474545-zrqgl"] Jul 13 17:43:28.137195 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:28.137162 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c79607c-1d48-4bea-b676-6f23cf5ad130" path="/var/lib/kubelet/pods/4c79607c-1d48-4bea-b676-6f23cf5ad130/volumes" Jul 13 17:43:28.137577 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:28.137550 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8f87a25-5ec0-478f-b043-08b7635e5105" path="/var/lib/kubelet/pods/a8f87a25-5ec0-478f-b043-08b7635e5105/volumes" Jul 13 17:43:37.791043 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:37.791004 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-587790-xrnm7"] Jul 13 17:43:37.791522 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:37.791502 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c79607c-1d48-4bea-b676-6f23cf5ad130" containerName="main" Jul 13 17:43:37.791612 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:37.791524 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c79607c-1d48-4bea-b676-6f23cf5ad130" containerName="main" Jul 13 17:43:37.791612 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:37.791544 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8f87a25-5ec0-478f-b043-08b7635e5105" containerName="main" Jul 13 17:43:37.791612 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:37.791553 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8f87a25-5ec0-478f-b043-08b7635e5105" containerName="main" Jul 13 17:43:37.791759 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:37.791710 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8f87a25-5ec0-478f-b043-08b7635e5105" containerName="main" Jul 13 17:43:37.791759 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:37.791726 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c79607c-1d48-4bea-b676-6f23cf5ad130" containerName="main" Jul 13 17:43:37.795917 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:37.795895 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-587790-xrnm7" Jul 13 17:43:37.804108 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:37.803998 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 13 17:43:37.804108 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:37.804009 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k25z5\"" Jul 13 17:43:37.806831 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:37.806808 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-587790-xrnm7"] Jul 13 17:43:37.811222 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:37.811196 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/77eb804a-4260-4fb6-b6a9-1d7d305cef35-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-587790-xrnm7\" (UID: \"77eb804a-4260-4fb6-b6a9-1d7d305cef35\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-587790-xrnm7" Jul 13 17:43:37.912146 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:37.912114 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/77eb804a-4260-4fb6-b6a9-1d7d305cef35-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-587790-xrnm7\" (UID: \"77eb804a-4260-4fb6-b6a9-1d7d305cef35\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-587790-xrnm7" Jul 13 17:43:37.912728 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:37.912710 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/77eb804a-4260-4fb6-b6a9-1d7d305cef35-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-587790-xrnm7\" (UID: \"77eb804a-4260-4fb6-b6a9-1d7d305cef35\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-587790-xrnm7" Jul 13 17:43:38.113461 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:38.113382 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-587790-xrnm7" Jul 13 17:43:38.253118 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:38.253058 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-587790-xrnm7"] Jul 13 17:43:38.256082 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:43:38.256055 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77eb804a_4260_4fb6_b6a9_1d7d305cef35.slice/crio-5096f963c967912747599edf6ab2c1e95a1e77e427e596829596b120348bd445 WatchSource:0}: Error finding container 5096f963c967912747599edf6ab2c1e95a1e77e427e596829596b120348bd445: Status 404 returned error can't find the container with id 5096f963c967912747599edf6ab2c1e95a1e77e427e596829596b120348bd445 Jul 13 17:43:38.258070 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:38.258048 2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 17:43:38.821618 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:38.821580 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-587790-xrnm7" event={"ID":"77eb804a-4260-4fb6-b6a9-1d7d305cef35","Type":"ContainerStarted","Data":"5096f963c967912747599edf6ab2c1e95a1e77e427e596829596b120348bd445"} Jul 13 17:43:41.837854 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:41.837820 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-587790-xrnm7" event={"ID":"77eb804a-4260-4fb6-b6a9-1d7d305cef35","Type":"ContainerStarted","Data":"7f4f5bbcfb5e731e86ec0c6f6af1f9705f21bf5d9010b486360f6f98faac7fc3"} Jul 13 17:43:41.856291 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:41.856250 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-587790-xrnm7" podStartSLOduration=1.6307865019999999 podStartE2EDuration="4.856234772s" totalImagesPullingTime="3.22544827s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:43:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:43:41.8535132 +0000 UTC m=+2318.251171580" watchObservedRunningTime="2026-07-13 17:43:41.856234772 +0000 UTC m=+2318.253893152" Jul 13 17:43:42.844263 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:42.844226 2569 generic.go:356] "Generic (PLEG): container finished" podID="77eb804a-4260-4fb6-b6a9-1d7d305cef35" containerID="7f4f5bbcfb5e731e86ec0c6f6af1f9705f21bf5d9010b486360f6f98faac7fc3" exitCode=0 Jul 13 17:43:42.844751 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:42.844312 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-587790-xrnm7" event={"ID":"77eb804a-4260-4fb6-b6a9-1d7d305cef35","Type":"ContainerDied","Data":"7f4f5bbcfb5e731e86ec0c6f6af1f9705f21bf5d9010b486360f6f98faac7fc3"} Jul 13 17:43:43.983222 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:43.983199 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-587790-xrnm7" Jul 13 17:43:44.054890 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:44.054863 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/77eb804a-4260-4fb6-b6a9-1d7d305cef35-maas-api-service-ca\") pod \"77eb804a-4260-4fb6-b6a9-1d7d305cef35\" (UID: \"77eb804a-4260-4fb6-b6a9-1d7d305cef35\") " Jul 13 17:43:44.055227 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:44.055201 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77eb804a-4260-4fb6-b6a9-1d7d305cef35-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "77eb804a-4260-4fb6-b6a9-1d7d305cef35" (UID: "77eb804a-4260-4fb6-b6a9-1d7d305cef35"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:43:44.155668 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:44.155644 2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/77eb804a-4260-4fb6-b6a9-1d7d305cef35-maas-api-service-ca\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:43:44.854709 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:44.854668 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-587790-xrnm7" event={"ID":"77eb804a-4260-4fb6-b6a9-1d7d305cef35","Type":"ContainerDied","Data":"5096f963c967912747599edf6ab2c1e95a1e77e427e596829596b120348bd445"} Jul 13 17:43:44.854709 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:44.854702 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-587790-xrnm7" Jul 13 17:43:44.854902 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:43:44.854729 2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5096f963c967912747599edf6ab2c1e95a1e77e427e596829596b120348bd445" Jul 13 17:44:28.479683 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:44:28.479639 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-edd1bf-xqwvb"] Jul 13 17:44:28.480183 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:44:28.480161 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77eb804a-4260-4fb6-b6a9-1d7d305cef35" containerName="revoke-keys" Jul 13 17:44:28.480238 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:44:28.480187 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="77eb804a-4260-4fb6-b6a9-1d7d305cef35" containerName="revoke-keys" Jul 13 17:44:28.480344 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:44:28.480330 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="77eb804a-4260-4fb6-b6a9-1d7d305cef35" containerName="revoke-keys" Jul 13 17:44:28.482391 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:44:28.482371 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-edd1bf-xqwvb" Jul 13 17:44:28.484652 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:44:28.484631 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k25z5\"" Jul 13 17:44:28.484774 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:44:28.484673 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 13 17:44:28.490943 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:44:28.490921 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-edd1bf-xqwvb"] Jul 13 17:44:28.623800 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:44:28.623772 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b07c283-6c14-48f6-97ba-a5ce2f12529d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-edd1bf-xqwvb\" (UID: \"9b07c283-6c14-48f6-97ba-a5ce2f12529d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-edd1bf-xqwvb" Jul 13 17:44:28.724330 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:44:28.724298 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b07c283-6c14-48f6-97ba-a5ce2f12529d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-edd1bf-xqwvb\" (UID: \"9b07c283-6c14-48f6-97ba-a5ce2f12529d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-edd1bf-xqwvb" Jul 13 17:44:28.724918 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:44:28.724897 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b07c283-6c14-48f6-97ba-a5ce2f12529d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-edd1bf-xqwvb\" (UID: \"9b07c283-6c14-48f6-97ba-a5ce2f12529d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-edd1bf-xqwvb" Jul 13 17:44:28.793841 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:44:28.793816 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-edd1bf-xqwvb" Jul 13 17:44:28.927544 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:44:28.927518 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-edd1bf-xqwvb"] Jul 13 17:44:28.929821 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:44:28.929783 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b07c283_6c14_48f6_97ba_a5ce2f12529d.slice/crio-2109eb21ea463d73de14a8420f743a17ddf63236076cf5ac78ac66af5eabf474 WatchSource:0}: Error finding container 2109eb21ea463d73de14a8420f743a17ddf63236076cf5ac78ac66af5eabf474: Status 404 returned error can't find the container with id 2109eb21ea463d73de14a8420f743a17ddf63236076cf5ac78ac66af5eabf474 Jul 13 17:44:29.076822 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:44:29.076754 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-edd1bf-xqwvb" event={"ID":"9b07c283-6c14-48f6-97ba-a5ce2f12529d","Type":"ContainerStarted","Data":"2109eb21ea463d73de14a8420f743a17ddf63236076cf5ac78ac66af5eabf474"} Jul 13 17:44:30.082676 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:44:30.082639 2569 generic.go:356] "Generic (PLEG): container finished" podID="9b07c283-6c14-48f6-97ba-a5ce2f12529d" containerID="7df39802474d966605aac454716205c96d225595fe0ddc51d3c9350195fd971b" exitCode=0 Jul 13 17:44:30.083099 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:44:30.082724 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-edd1bf-xqwvb" event={"ID":"9b07c283-6c14-48f6-97ba-a5ce2f12529d","Type":"ContainerDied","Data":"7df39802474d966605aac454716205c96d225595fe0ddc51d3c9350195fd971b"} Jul 13 17:44:31.220492 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:44:31.220469 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-edd1bf-xqwvb" Jul 13 17:44:31.346253 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:44:31.346180 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b07c283-6c14-48f6-97ba-a5ce2f12529d-maas-api-service-ca\") pod \"9b07c283-6c14-48f6-97ba-a5ce2f12529d\" (UID: \"9b07c283-6c14-48f6-97ba-a5ce2f12529d\") " Jul 13 17:44:31.346516 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:44:31.346490 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b07c283-6c14-48f6-97ba-a5ce2f12529d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9b07c283-6c14-48f6-97ba-a5ce2f12529d" (UID: "9b07c283-6c14-48f6-97ba-a5ce2f12529d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:44:31.447377 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:44:31.447351 2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b07c283-6c14-48f6-97ba-a5ce2f12529d-maas-api-service-ca\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:44:32.093453 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:44:32.093427 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-edd1bf-xqwvb" Jul 13 17:44:32.093607 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:44:32.093423 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-edd1bf-xqwvb" event={"ID":"9b07c283-6c14-48f6-97ba-a5ce2f12529d","Type":"ContainerDied","Data":"2109eb21ea463d73de14a8420f743a17ddf63236076cf5ac78ac66af5eabf474"} Jul 13 17:44:32.093607 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:44:32.093598 2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2109eb21ea463d73de14a8420f743a17ddf63236076cf5ac78ac66af5eabf474" Jul 13 17:44:54.766918 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:44:54.766882 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e386c8-s2w6c"] Jul 13 17:44:54.767308 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:44:54.767264 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b07c283-6c14-48f6-97ba-a5ce2f12529d" containerName="revoke-keys" Jul 13 17:44:54.767308 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:44:54.767276 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b07c283-6c14-48f6-97ba-a5ce2f12529d" containerName="revoke-keys" Jul 13 17:44:54.767415 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:44:54.767339 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b07c283-6c14-48f6-97ba-a5ce2f12529d" containerName="revoke-keys" Jul 13 17:44:54.769297 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:44:54.769280 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e386c8-s2w6c" Jul 13 17:44:54.774859 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:44:54.774832 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k25z5\"" Jul 13 17:44:54.774982 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:44:54.774960 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 13 17:44:54.782217 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:44:54.782194 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e386c8-s2w6c"] Jul 13 17:44:54.827955 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:44:54.827929 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/589b44c1-8180-4474-89a6-e44a6e8fd53f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-e386c8-s2w6c\" (UID: \"589b44c1-8180-4474-89a6-e44a6e8fd53f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e386c8-s2w6c" Jul 13 17:44:54.929120 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:44:54.929091 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/589b44c1-8180-4474-89a6-e44a6e8fd53f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-e386c8-s2w6c\" (UID: \"589b44c1-8180-4474-89a6-e44a6e8fd53f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e386c8-s2w6c" Jul 13 17:44:54.929713 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:44:54.929692 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/589b44c1-8180-4474-89a6-e44a6e8fd53f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-e386c8-s2w6c\" (UID: \"589b44c1-8180-4474-89a6-e44a6e8fd53f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e386c8-s2w6c" Jul 13 17:44:55.079621 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:44:55.079586 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e386c8-s2w6c" Jul 13 17:44:55.210669 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:44:55.210645 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e386c8-s2w6c"] Jul 13 17:44:55.213039 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:44:55.213011 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod589b44c1_8180_4474_89a6_e44a6e8fd53f.slice/crio-7bac3f5fb4043edcf0be85a9ff7929002396795eecbca5890e0cfcd6e0c16e48 WatchSource:0}: Error finding container 7bac3f5fb4043edcf0be85a9ff7929002396795eecbca5890e0cfcd6e0c16e48: Status 404 returned error can't find the container with id 7bac3f5fb4043edcf0be85a9ff7929002396795eecbca5890e0cfcd6e0c16e48 Jul 13 17:44:56.208416 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:44:56.208333 2569 generic.go:356] "Generic (PLEG): container finished" podID="589b44c1-8180-4474-89a6-e44a6e8fd53f" containerID="24d0e76dd6ecc67a7ebb49b601da90d738fa906ec753c72bca5826d3c40ba854" exitCode=0 Jul 13 17:44:56.208769 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:44:56.208400 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e386c8-s2w6c" event={"ID":"589b44c1-8180-4474-89a6-e44a6e8fd53f","Type":"ContainerDied","Data":"24d0e76dd6ecc67a7ebb49b601da90d738fa906ec753c72bca5826d3c40ba854"} Jul 13 17:44:56.208769 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:44:56.208458 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e386c8-s2w6c" event={"ID":"589b44c1-8180-4474-89a6-e44a6e8fd53f","Type":"ContainerStarted","Data":"7bac3f5fb4043edcf0be85a9ff7929002396795eecbca5890e0cfcd6e0c16e48"} Jul 13 17:44:57.342167 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:44:57.342146 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e386c8-s2w6c" Jul 13 17:44:57.454169 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:44:57.454140 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/589b44c1-8180-4474-89a6-e44a6e8fd53f-maas-api-service-ca\") pod \"589b44c1-8180-4474-89a6-e44a6e8fd53f\" (UID: \"589b44c1-8180-4474-89a6-e44a6e8fd53f\") " Jul 13 17:44:57.454468 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:44:57.454444 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/589b44c1-8180-4474-89a6-e44a6e8fd53f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "589b44c1-8180-4474-89a6-e44a6e8fd53f" (UID: "589b44c1-8180-4474-89a6-e44a6e8fd53f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:44:57.555007 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:44:57.554981 2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/589b44c1-8180-4474-89a6-e44a6e8fd53f-maas-api-service-ca\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:44:58.219098 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:44:58.219063 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e386c8-s2w6c" event={"ID":"589b44c1-8180-4474-89a6-e44a6e8fd53f","Type":"ContainerDied","Data":"7bac3f5fb4043edcf0be85a9ff7929002396795eecbca5890e0cfcd6e0c16e48"} Jul 13 17:44:58.219224 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:44:58.219115 2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7bac3f5fb4043edcf0be85a9ff7929002396795eecbca5890e0cfcd6e0c16e48" Jul 13 17:44:58.219224 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:44:58.219082 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e386c8-s2w6c" Jul 13 17:45:00.143171 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:00.143139 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29732745-2x6fr"] Jul 13 17:45:00.143525 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:00.143483 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="589b44c1-8180-4474-89a6-e44a6e8fd53f" containerName="revoke-keys" Jul 13 17:45:00.143525 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:00.143494 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="589b44c1-8180-4474-89a6-e44a6e8fd53f" containerName="revoke-keys" Jul 13 17:45:00.143622 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:00.143589 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="589b44c1-8180-4474-89a6-e44a6e8fd53f" containerName="revoke-keys" Jul 13 17:45:00.145681 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:00.145658 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732745-2x6fr" Jul 13 17:45:00.147899 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:00.147877 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-e386c8-29732745-qz5bk"] Jul 13 17:45:00.148468 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:00.148450 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k25z5\"" Jul 13 17:45:00.150179 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:00.150163 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-e386c8-29732745-qz5bk" Jul 13 17:45:00.155230 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:00.155206 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29732745-2x6fr"] Jul 13 17:45:00.164033 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:00.162321 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-e386c8-29732745-qz5bk"] Jul 13 17:45:00.164490 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:00.164463 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732745-2x6fr" Jul 13 17:45:00.172816 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:00.172794 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-e386c8-29732745-qz5bk" Jul 13 17:45:00.519920 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:00.519893 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29732745-2x6fr"] Jul 13 17:45:00.521816 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:45:00.521779 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1dce35e_1d10_48d9_ab31_ca7014c4d960.slice/crio-36e3bdbe8db762453a36179c81d23663706dbc89981ebc0f6e7eeec76215833e WatchSource:0}: Error finding container 36e3bdbe8db762453a36179c81d23663706dbc89981ebc0f6e7eeec76215833e: Status 404 returned error can't find the container with id 36e3bdbe8db762453a36179c81d23663706dbc89981ebc0f6e7eeec76215833e Jul 13 17:45:00.546428 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:00.546393 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-e386c8-29732745-qz5bk"] Jul 13 17:45:00.555622 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:45:00.555598 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06a7aa2b_3422_4dc4_b0d0_7b4969a88296.slice/crio-31d663d69bef68d8a52fb88cffc4a89fa8db72cf8f5f07d0f5f702b96afbc029 WatchSource:0}: Error finding container 31d663d69bef68d8a52fb88cffc4a89fa8db72cf8f5f07d0f5f702b96afbc029: Status 404 returned error can't find the container with id 31d663d69bef68d8a52fb88cffc4a89fa8db72cf8f5f07d0f5f702b96afbc029 Jul 13 17:45:01.241811 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:01.241724 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-e386c8-29732745-qz5bk" event={"ID":"06a7aa2b-3422-4dc4-b0d0-7b4969a88296","Type":"ContainerStarted","Data":"ec854f28e424e2b1308fe3ff078214c6d76e4a261bc8cbc5b92740393740fd1e"} Jul 13 17:45:01.242183 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:01.241828 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-e386c8-29732745-qz5bk" event={"ID":"06a7aa2b-3422-4dc4-b0d0-7b4969a88296","Type":"ContainerStarted","Data":"31d663d69bef68d8a52fb88cffc4a89fa8db72cf8f5f07d0f5f702b96afbc029"} Jul 13 17:45:01.243225 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:01.243199 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732745-2x6fr" event={"ID":"e1dce35e-1d10-48d9-ab31-ca7014c4d960","Type":"ContainerStarted","Data":"09ea015370e061dc329a6d25b40941df70454a852e57d512c739bff6d984000f"} Jul 13 17:45:01.243348 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:01.243244 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732745-2x6fr" event={"ID":"e1dce35e-1d10-48d9-ab31-ca7014c4d960","Type":"ContainerStarted","Data":"36e3bdbe8db762453a36179c81d23663706dbc89981ebc0f6e7eeec76215833e"} Jul 13 17:45:01.259690 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:01.259648 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-e386c8-29732745-qz5bk" podStartSLOduration=1.259635634 podStartE2EDuration="1.259635634s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:45:01.257544667 +0000 UTC m=+2397.655203046" watchObservedRunningTime="2026-07-13 17:45:01.259635634 +0000 UTC m=+2397.657294004" Jul 13 17:45:01.275089 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:01.275051 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732745-2x6fr" podStartSLOduration=1.2750370690000001 podStartE2EDuration="1.275037069s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:45:01.27311161 +0000 UTC m=+2397.670769990" watchObservedRunningTime="2026-07-13 17:45:01.275037069 +0000 UTC m=+2397.672695497" Jul 13 17:45:02.250485 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:02.250454 2569 generic.go:356] "Generic (PLEG): container finished" podID="06a7aa2b-3422-4dc4-b0d0-7b4969a88296" containerID="ec854f28e424e2b1308fe3ff078214c6d76e4a261bc8cbc5b92740393740fd1e" exitCode=0 Jul 13 17:45:02.250901 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:02.250533 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-e386c8-29732745-qz5bk" event={"ID":"06a7aa2b-3422-4dc4-b0d0-7b4969a88296","Type":"ContainerDied","Data":"ec854f28e424e2b1308fe3ff078214c6d76e4a261bc8cbc5b92740393740fd1e"} Jul 13 17:45:02.252031 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:02.252008 2569 generic.go:356] "Generic (PLEG): container finished" podID="e1dce35e-1d10-48d9-ab31-ca7014c4d960" containerID="09ea015370e061dc329a6d25b40941df70454a852e57d512c739bff6d984000f" exitCode=0 Jul 13 17:45:02.252133 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:02.252065 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732745-2x6fr" event={"ID":"e1dce35e-1d10-48d9-ab31-ca7014c4d960","Type":"ContainerDied","Data":"09ea015370e061dc329a6d25b40941df70454a852e57d512c739bff6d984000f"} Jul 13 17:45:03.404316 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:03.404287 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732745-2x6fr" Jul 13 17:45:03.407057 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:03.407039 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-e386c8-29732745-qz5bk" Jul 13 17:45:04.243758 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:04.243729 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-hzcql_c40e6e88-ab3f-4e3c-a0e7-2400bda333eb/console-operator/1.log" Jul 13 17:45:04.252349 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:04.252320 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q2pnb_6a531b70-4a58-4752-8fa8-5ba634b542ea/ovn-acl-logging/0.log" Jul 13 17:45:04.253641 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:04.253621 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-hzcql_c40e6e88-ab3f-4e3c-a0e7-2400bda333eb/console-operator/1.log" Jul 13 17:45:04.262246 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:04.262219 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-e386c8-29732745-qz5bk" event={"ID":"06a7aa2b-3422-4dc4-b0d0-7b4969a88296","Type":"ContainerDied","Data":"31d663d69bef68d8a52fb88cffc4a89fa8db72cf8f5f07d0f5f702b96afbc029"} Jul 13 17:45:04.262246 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:04.262227 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-e386c8-29732745-qz5bk" Jul 13 17:45:04.262387 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:04.262260 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q2pnb_6a531b70-4a58-4752-8fa8-5ba634b542ea/ovn-acl-logging/0.log" Jul 13 17:45:04.262387 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:04.262269 2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="31d663d69bef68d8a52fb88cffc4a89fa8db72cf8f5f07d0f5f702b96afbc029" Jul 13 17:45:04.263686 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:04.263667 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732745-2x6fr" Jul 13 17:45:04.263805 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:04.263670 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732745-2x6fr" event={"ID":"e1dce35e-1d10-48d9-ab31-ca7014c4d960","Type":"ContainerDied","Data":"36e3bdbe8db762453a36179c81d23663706dbc89981ebc0f6e7eeec76215833e"} Jul 13 17:45:04.263871 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:04.263851 2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="36e3bdbe8db762453a36179c81d23663706dbc89981ebc0f6e7eeec76215833e" Jul 13 17:45:05.354510 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:05.354473 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-e386c8-29732745-qz5bk"] Jul 13 17:45:05.358027 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:05.358004 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-e386c8-29732745-qz5bk"] Jul 13 17:45:06.137721 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:06.137687 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06a7aa2b-3422-4dc4-b0d0-7b4969a88296" path="/var/lib/kubelet/pods/06a7aa2b-3422-4dc4-b0d0-7b4969a88296/volumes" Jul 13 17:45:51.331681 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:51.331649 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-714452/auth-test-model-714452-kserve-7fbf74694c-j4q8q"] Jul 13 17:45:51.332244 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:51.331985 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06a7aa2b-3422-4dc4-b0d0-7b4969a88296" containerName="cleanup" Jul 13 17:45:51.332244 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:51.331995 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="06a7aa2b-3422-4dc4-b0d0-7b4969a88296" containerName="cleanup" Jul 13 17:45:51.332244 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:51.332006 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1dce35e-1d10-48d9-ab31-ca7014c4d960" containerName="cleanup" Jul 13 17:45:51.332244 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:51.332012 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1dce35e-1d10-48d9-ab31-ca7014c4d960" containerName="cleanup" Jul 13 17:45:51.332244 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:51.332077 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="06a7aa2b-3422-4dc4-b0d0-7b4969a88296" containerName="cleanup" Jul 13 17:45:51.332244 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:51.332089 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1dce35e-1d10-48d9-ab31-ca7014c4d960" containerName="cleanup" Jul 13 17:45:51.334515 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:51.334496 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-714452/auth-test-model-714452-kserve-7fbf74694c-j4q8q" Jul 13 17:45:51.337737 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:51.337720 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-714452\"/\"auth-test-model-714452-kserve-self-signed-certs\"" Jul 13 17:45:51.337945 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:51.337725 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-714452\"/\"default-dockercfg-s6hqf\"" Jul 13 17:45:51.338053 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:51.337758 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-714452\"/\"kube-root-ca.crt\"" Jul 13 17:45:51.339239 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:51.339222 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-714452\"/\"openshift-service-ca.crt\"" Jul 13 17:45:51.347609 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:51.347587 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-714452/auth-test-model-714452-kserve-7fbf74694c-j4q8q"] Jul 13 17:45:51.485788 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:51.485753 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ab939d8-d4d3-4b1f-bb02-8c8670c63c03-home\") pod \"auth-test-model-714452-kserve-7fbf74694c-j4q8q\" (UID: \"3ab939d8-d4d3-4b1f-bb02-8c8670c63c03\") " pod="ai-tenant-e2e-shared-a-714452/auth-test-model-714452-kserve-7fbf74694c-j4q8q" Jul 13 17:45:51.485788 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:51.485790 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ab939d8-d4d3-4b1f-bb02-8c8670c63c03-dshm\") pod \"auth-test-model-714452-kserve-7fbf74694c-j4q8q\" (UID: \"3ab939d8-d4d3-4b1f-bb02-8c8670c63c03\") " pod="ai-tenant-e2e-shared-a-714452/auth-test-model-714452-kserve-7fbf74694c-j4q8q" Jul 13 17:45:51.485963 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:51.485892 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ab939d8-d4d3-4b1f-bb02-8c8670c63c03-tmp-dir\") pod \"auth-test-model-714452-kserve-7fbf74694c-j4q8q\" (UID: \"3ab939d8-d4d3-4b1f-bb02-8c8670c63c03\") " pod="ai-tenant-e2e-shared-a-714452/auth-test-model-714452-kserve-7fbf74694c-j4q8q" Jul 13 17:45:51.485963 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:51.485926 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ab939d8-d4d3-4b1f-bb02-8c8670c63c03-model-cache\") pod \"auth-test-model-714452-kserve-7fbf74694c-j4q8q\" (UID: \"3ab939d8-d4d3-4b1f-bb02-8c8670c63c03\") " pod="ai-tenant-e2e-shared-a-714452/auth-test-model-714452-kserve-7fbf74694c-j4q8q" Jul 13 17:45:51.486032 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:51.485973 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ab939d8-d4d3-4b1f-bb02-8c8670c63c03-tls-certs\") pod \"auth-test-model-714452-kserve-7fbf74694c-j4q8q\" (UID: \"3ab939d8-d4d3-4b1f-bb02-8c8670c63c03\") " pod="ai-tenant-e2e-shared-a-714452/auth-test-model-714452-kserve-7fbf74694c-j4q8q" Jul 13 17:45:51.486032 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:51.486004 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlsgr\" (UniqueName: \"kubernetes.io/projected/3ab939d8-d4d3-4b1f-bb02-8c8670c63c03-kube-api-access-tlsgr\") pod \"auth-test-model-714452-kserve-7fbf74694c-j4q8q\" (UID: \"3ab939d8-d4d3-4b1f-bb02-8c8670c63c03\") " pod="ai-tenant-e2e-shared-a-714452/auth-test-model-714452-kserve-7fbf74694c-j4q8q" Jul 13 17:45:51.587420 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:51.587352 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ab939d8-d4d3-4b1f-bb02-8c8670c63c03-tmp-dir\") pod \"auth-test-model-714452-kserve-7fbf74694c-j4q8q\" (UID: \"3ab939d8-d4d3-4b1f-bb02-8c8670c63c03\") " pod="ai-tenant-e2e-shared-a-714452/auth-test-model-714452-kserve-7fbf74694c-j4q8q" Jul 13 17:45:51.587420 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:51.587385 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ab939d8-d4d3-4b1f-bb02-8c8670c63c03-model-cache\") pod \"auth-test-model-714452-kserve-7fbf74694c-j4q8q\" (UID: \"3ab939d8-d4d3-4b1f-bb02-8c8670c63c03\") " pod="ai-tenant-e2e-shared-a-714452/auth-test-model-714452-kserve-7fbf74694c-j4q8q" Jul 13 17:45:51.587420 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:51.587416 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ab939d8-d4d3-4b1f-bb02-8c8670c63c03-tls-certs\") pod \"auth-test-model-714452-kserve-7fbf74694c-j4q8q\" (UID: \"3ab939d8-d4d3-4b1f-bb02-8c8670c63c03\") " pod="ai-tenant-e2e-shared-a-714452/auth-test-model-714452-kserve-7fbf74694c-j4q8q" Jul 13 17:45:51.587670 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:51.587447 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlsgr\" (UniqueName: \"kubernetes.io/projected/3ab939d8-d4d3-4b1f-bb02-8c8670c63c03-kube-api-access-tlsgr\") pod \"auth-test-model-714452-kserve-7fbf74694c-j4q8q\" (UID: \"3ab939d8-d4d3-4b1f-bb02-8c8670c63c03\") " pod="ai-tenant-e2e-shared-a-714452/auth-test-model-714452-kserve-7fbf74694c-j4q8q" Jul 13 17:45:51.587670 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:51.587486 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ab939d8-d4d3-4b1f-bb02-8c8670c63c03-home\") pod \"auth-test-model-714452-kserve-7fbf74694c-j4q8q\" (UID: \"3ab939d8-d4d3-4b1f-bb02-8c8670c63c03\") " pod="ai-tenant-e2e-shared-a-714452/auth-test-model-714452-kserve-7fbf74694c-j4q8q" Jul 13 17:45:51.587670 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:51.587505 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ab939d8-d4d3-4b1f-bb02-8c8670c63c03-dshm\") pod \"auth-test-model-714452-kserve-7fbf74694c-j4q8q\" (UID: \"3ab939d8-d4d3-4b1f-bb02-8c8670c63c03\") " pod="ai-tenant-e2e-shared-a-714452/auth-test-model-714452-kserve-7fbf74694c-j4q8q" Jul 13 17:45:51.587893 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:51.587844 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ab939d8-d4d3-4b1f-bb02-8c8670c63c03-model-cache\") pod \"auth-test-model-714452-kserve-7fbf74694c-j4q8q\" (UID: \"3ab939d8-d4d3-4b1f-bb02-8c8670c63c03\") " pod="ai-tenant-e2e-shared-a-714452/auth-test-model-714452-kserve-7fbf74694c-j4q8q" Jul 13 17:45:51.588015 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:51.587895 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ab939d8-d4d3-4b1f-bb02-8c8670c63c03-tmp-dir\") pod \"auth-test-model-714452-kserve-7fbf74694c-j4q8q\" (UID: \"3ab939d8-d4d3-4b1f-bb02-8c8670c63c03\") " pod="ai-tenant-e2e-shared-a-714452/auth-test-model-714452-kserve-7fbf74694c-j4q8q" Jul 13 17:45:51.588015 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:51.587993 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ab939d8-d4d3-4b1f-bb02-8c8670c63c03-home\") pod \"auth-test-model-714452-kserve-7fbf74694c-j4q8q\" (UID: \"3ab939d8-d4d3-4b1f-bb02-8c8670c63c03\") " pod="ai-tenant-e2e-shared-a-714452/auth-test-model-714452-kserve-7fbf74694c-j4q8q" Jul 13 17:45:51.589809 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:51.589783 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ab939d8-d4d3-4b1f-bb02-8c8670c63c03-dshm\") pod \"auth-test-model-714452-kserve-7fbf74694c-j4q8q\" (UID: \"3ab939d8-d4d3-4b1f-bb02-8c8670c63c03\") " pod="ai-tenant-e2e-shared-a-714452/auth-test-model-714452-kserve-7fbf74694c-j4q8q" Jul 13 17:45:51.590076 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:51.590055 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ab939d8-d4d3-4b1f-bb02-8c8670c63c03-tls-certs\") pod \"auth-test-model-714452-kserve-7fbf74694c-j4q8q\" (UID: \"3ab939d8-d4d3-4b1f-bb02-8c8670c63c03\") " pod="ai-tenant-e2e-shared-a-714452/auth-test-model-714452-kserve-7fbf74694c-j4q8q" Jul 13 17:45:51.599600 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:51.599553 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlsgr\" (UniqueName: \"kubernetes.io/projected/3ab939d8-d4d3-4b1f-bb02-8c8670c63c03-kube-api-access-tlsgr\") pod \"auth-test-model-714452-kserve-7fbf74694c-j4q8q\" (UID: \"3ab939d8-d4d3-4b1f-bb02-8c8670c63c03\") " pod="ai-tenant-e2e-shared-a-714452/auth-test-model-714452-kserve-7fbf74694c-j4q8q" Jul 13 17:45:51.645085 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:51.645064 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-714452/auth-test-model-714452-kserve-7fbf74694c-j4q8q" Jul 13 17:45:51.777170 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:51.776995 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-714452/auth-test-model-714452-kserve-7fbf74694c-j4q8q"] Jul 13 17:45:51.779342 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:45:51.779314 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3ab939d8_d4d3_4b1f_bb02_8c8670c63c03.slice/crio-9e8d125878230c81e04c6b2b57d525d94d3de573131d39abedae80517c9e218f WatchSource:0}: Error finding container 9e8d125878230c81e04c6b2b57d525d94d3de573131d39abedae80517c9e218f: Status 404 returned error can't find the container with id 9e8d125878230c81e04c6b2b57d525d94d3de573131d39abedae80517c9e218f Jul 13 17:45:52.507715 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:52.507680 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-714452/auth-test-model-714452-kserve-7fbf74694c-j4q8q" event={"ID":"3ab939d8-d4d3-4b1f-bb02-8c8670c63c03","Type":"ContainerStarted","Data":"38d6648d5a491a9bec3e90b22392cfd18bc48719e86b9d7e0dc801f1799123e2"} Jul 13 17:45:52.508113 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:52.507725 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-714452/auth-test-model-714452-kserve-7fbf74694c-j4q8q" event={"ID":"3ab939d8-d4d3-4b1f-bb02-8c8670c63c03","Type":"ContainerStarted","Data":"9e8d125878230c81e04c6b2b57d525d94d3de573131d39abedae80517c9e218f"} Jul 13 17:45:52.527618 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:45:52.527535 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-714452/auth-test-model-714452-kserve-7fbf74694c-j4q8q" podStartSLOduration=1.527520851 podStartE2EDuration="1.527520851s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:45:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:45:52.526000895 +0000 UTC m=+2448.923659273" watchObservedRunningTime="2026-07-13 17:45:52.527520851 +0000 UTC m=+2448.925179230" Jul 13 17:46:01.645914 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:46:01.645877 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-714452/auth-test-model-714452-kserve-7fbf74694c-j4q8q" Jul 13 17:46:01.646344 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:46:01.645963 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-714452/auth-test-model-714452-kserve-7fbf74694c-j4q8q" Jul 13 17:46:01.658179 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:46:01.658160 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-714452/auth-test-model-714452-kserve-7fbf74694c-j4q8q" Jul 13 17:46:02.569056 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:46:02.569023 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-714452/auth-test-model-714452-kserve-7fbf74694c-j4q8q" Jul 13 17:46:33.147363 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:46:33.147329 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-31a784/auth-test-model-31a784-kserve-795957db7-gcbh8"] Jul 13 17:46:33.149678 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:46:33.149661 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-31a784/auth-test-model-31a784-kserve-795957db7-gcbh8" Jul 13 17:46:33.152950 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:46:33.152924 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-31a784\"/\"kube-root-ca.crt\"" Jul 13 17:46:33.153624 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:46:33.153603 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-31a784\"/\"default-dockercfg-d2rx8\"" Jul 13 17:46:33.153745 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:46:33.153625 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-31a784\"/\"openshift-service-ca.crt\"" Jul 13 17:46:33.153745 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:46:33.153635 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-31a784\"/\"auth-test-model-31a784-kserve-self-signed-certs\"" Jul 13 17:46:33.164855 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:46:33.164834 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-31a784/auth-test-model-31a784-kserve-795957db7-gcbh8"] Jul 13 17:46:33.234882 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:46:33.234856 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef047c6e-dd16-4a46-bd3e-54ed872ab4c3-tls-certs\") pod \"auth-test-model-31a784-kserve-795957db7-gcbh8\" (UID: \"ef047c6e-dd16-4a46-bd3e-54ed872ab4c3\") " pod="ai-tenant-e2e-shared-b-31a784/auth-test-model-31a784-kserve-795957db7-gcbh8" Jul 13 17:46:33.235014 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:46:33.234888 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgjj7\" (UniqueName: \"kubernetes.io/projected/ef047c6e-dd16-4a46-bd3e-54ed872ab4c3-kube-api-access-wgjj7\") pod \"auth-test-model-31a784-kserve-795957db7-gcbh8\" (UID: \"ef047c6e-dd16-4a46-bd3e-54ed872ab4c3\") " pod="ai-tenant-e2e-shared-b-31a784/auth-test-model-31a784-kserve-795957db7-gcbh8" Jul 13 17:46:33.235014 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:46:33.234912 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef047c6e-dd16-4a46-bd3e-54ed872ab4c3-model-cache\") pod \"auth-test-model-31a784-kserve-795957db7-gcbh8\" (UID: \"ef047c6e-dd16-4a46-bd3e-54ed872ab4c3\") " pod="ai-tenant-e2e-shared-b-31a784/auth-test-model-31a784-kserve-795957db7-gcbh8" Jul 13 17:46:33.235089 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:46:33.235052 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef047c6e-dd16-4a46-bd3e-54ed872ab4c3-dshm\") pod \"auth-test-model-31a784-kserve-795957db7-gcbh8\" (UID: \"ef047c6e-dd16-4a46-bd3e-54ed872ab4c3\") " pod="ai-tenant-e2e-shared-b-31a784/auth-test-model-31a784-kserve-795957db7-gcbh8" Jul 13 17:46:33.235166 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:46:33.235152 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef047c6e-dd16-4a46-bd3e-54ed872ab4c3-home\") pod \"auth-test-model-31a784-kserve-795957db7-gcbh8\" (UID: \"ef047c6e-dd16-4a46-bd3e-54ed872ab4c3\") " pod="ai-tenant-e2e-shared-b-31a784/auth-test-model-31a784-kserve-795957db7-gcbh8" Jul 13 17:46:33.235201 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:46:33.235176 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef047c6e-dd16-4a46-bd3e-54ed872ab4c3-tmp-dir\") pod \"auth-test-model-31a784-kserve-795957db7-gcbh8\" (UID: \"ef047c6e-dd16-4a46-bd3e-54ed872ab4c3\") " pod="ai-tenant-e2e-shared-b-31a784/auth-test-model-31a784-kserve-795957db7-gcbh8" Jul 13 17:46:33.336192 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:46:33.336163 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef047c6e-dd16-4a46-bd3e-54ed872ab4c3-dshm\") pod \"auth-test-model-31a784-kserve-795957db7-gcbh8\" (UID: \"ef047c6e-dd16-4a46-bd3e-54ed872ab4c3\") " pod="ai-tenant-e2e-shared-b-31a784/auth-test-model-31a784-kserve-795957db7-gcbh8" Jul 13 17:46:33.336313 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:46:33.336215 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef047c6e-dd16-4a46-bd3e-54ed872ab4c3-home\") pod \"auth-test-model-31a784-kserve-795957db7-gcbh8\" (UID: \"ef047c6e-dd16-4a46-bd3e-54ed872ab4c3\") " pod="ai-tenant-e2e-shared-b-31a784/auth-test-model-31a784-kserve-795957db7-gcbh8" Jul 13 17:46:33.336313 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:46:33.336231 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef047c6e-dd16-4a46-bd3e-54ed872ab4c3-tmp-dir\") pod \"auth-test-model-31a784-kserve-795957db7-gcbh8\" (UID: \"ef047c6e-dd16-4a46-bd3e-54ed872ab4c3\") " pod="ai-tenant-e2e-shared-b-31a784/auth-test-model-31a784-kserve-795957db7-gcbh8" Jul 13 17:46:33.336313 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:46:33.336256 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef047c6e-dd16-4a46-bd3e-54ed872ab4c3-tls-certs\") pod \"auth-test-model-31a784-kserve-795957db7-gcbh8\" (UID: \"ef047c6e-dd16-4a46-bd3e-54ed872ab4c3\") " pod="ai-tenant-e2e-shared-b-31a784/auth-test-model-31a784-kserve-795957db7-gcbh8" Jul 13 17:46:33.336313 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:46:33.336271 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgjj7\" (UniqueName: \"kubernetes.io/projected/ef047c6e-dd16-4a46-bd3e-54ed872ab4c3-kube-api-access-wgjj7\") pod \"auth-test-model-31a784-kserve-795957db7-gcbh8\" (UID: \"ef047c6e-dd16-4a46-bd3e-54ed872ab4c3\") " pod="ai-tenant-e2e-shared-b-31a784/auth-test-model-31a784-kserve-795957db7-gcbh8" Jul 13 17:46:33.336313 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:46:33.336292 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef047c6e-dd16-4a46-bd3e-54ed872ab4c3-model-cache\") pod \"auth-test-model-31a784-kserve-795957db7-gcbh8\" (UID: \"ef047c6e-dd16-4a46-bd3e-54ed872ab4c3\") " pod="ai-tenant-e2e-shared-b-31a784/auth-test-model-31a784-kserve-795957db7-gcbh8" Jul 13 17:46:33.336661 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:46:33.336643 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef047c6e-dd16-4a46-bd3e-54ed872ab4c3-model-cache\") pod \"auth-test-model-31a784-kserve-795957db7-gcbh8\" (UID: \"ef047c6e-dd16-4a46-bd3e-54ed872ab4c3\") " pod="ai-tenant-e2e-shared-b-31a784/auth-test-model-31a784-kserve-795957db7-gcbh8" Jul 13 17:46:33.336738 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:46:33.336702 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef047c6e-dd16-4a46-bd3e-54ed872ab4c3-tmp-dir\") pod \"auth-test-model-31a784-kserve-795957db7-gcbh8\" (UID: \"ef047c6e-dd16-4a46-bd3e-54ed872ab4c3\") " pod="ai-tenant-e2e-shared-b-31a784/auth-test-model-31a784-kserve-795957db7-gcbh8" Jul 13 17:46:33.336791 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:46:33.336740 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef047c6e-dd16-4a46-bd3e-54ed872ab4c3-home\") pod \"auth-test-model-31a784-kserve-795957db7-gcbh8\" (UID: \"ef047c6e-dd16-4a46-bd3e-54ed872ab4c3\") " pod="ai-tenant-e2e-shared-b-31a784/auth-test-model-31a784-kserve-795957db7-gcbh8" Jul 13 17:46:33.338286 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:46:33.338268 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef047c6e-dd16-4a46-bd3e-54ed872ab4c3-dshm\") pod \"auth-test-model-31a784-kserve-795957db7-gcbh8\" (UID: \"ef047c6e-dd16-4a46-bd3e-54ed872ab4c3\") " pod="ai-tenant-e2e-shared-b-31a784/auth-test-model-31a784-kserve-795957db7-gcbh8" Jul 13 17:46:33.338647 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:46:33.338631 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef047c6e-dd16-4a46-bd3e-54ed872ab4c3-tls-certs\") pod \"auth-test-model-31a784-kserve-795957db7-gcbh8\" (UID: \"ef047c6e-dd16-4a46-bd3e-54ed872ab4c3\") " pod="ai-tenant-e2e-shared-b-31a784/auth-test-model-31a784-kserve-795957db7-gcbh8" Jul 13 17:46:33.344118 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:46:33.344098 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgjj7\" (UniqueName: \"kubernetes.io/projected/ef047c6e-dd16-4a46-bd3e-54ed872ab4c3-kube-api-access-wgjj7\") pod \"auth-test-model-31a784-kserve-795957db7-gcbh8\" (UID: \"ef047c6e-dd16-4a46-bd3e-54ed872ab4c3\") " pod="ai-tenant-e2e-shared-b-31a784/auth-test-model-31a784-kserve-795957db7-gcbh8" Jul 13 17:46:33.459494 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:46:33.459431 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-31a784/auth-test-model-31a784-kserve-795957db7-gcbh8" Jul 13 17:46:33.627705 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:46:33.627670 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-31a784/auth-test-model-31a784-kserve-795957db7-gcbh8"] Jul 13 17:46:33.630991 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:46:33.630957 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podef047c6e_dd16_4a46_bd3e_54ed872ab4c3.slice/crio-e371a826839f77ea8e3b19d4b94f480060dc90b98dadcacb84d177d26f5a0e4d WatchSource:0}: Error finding container e371a826839f77ea8e3b19d4b94f480060dc90b98dadcacb84d177d26f5a0e4d: Status 404 returned error can't find the container with id e371a826839f77ea8e3b19d4b94f480060dc90b98dadcacb84d177d26f5a0e4d Jul 13 17:46:33.711179 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:46:33.711109 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-31a784/auth-test-model-31a784-kserve-795957db7-gcbh8" event={"ID":"ef047c6e-dd16-4a46-bd3e-54ed872ab4c3","Type":"ContainerStarted","Data":"085366bf27a8902953becb168737a68048ed7aa1c85ead7cf0046aa11e8e3532"} Jul 13 17:46:33.711179 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:46:33.711165 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-31a784/auth-test-model-31a784-kserve-795957db7-gcbh8" event={"ID":"ef047c6e-dd16-4a46-bd3e-54ed872ab4c3","Type":"ContainerStarted","Data":"e371a826839f77ea8e3b19d4b94f480060dc90b98dadcacb84d177d26f5a0e4d"} Jul 13 17:46:33.730958 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:46:33.730915 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-31a784/auth-test-model-31a784-kserve-795957db7-gcbh8" podStartSLOduration=0.730900216 podStartE2EDuration="730.900216ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:46:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:46:33.730503893 +0000 UTC m=+2490.128162273" watchObservedRunningTime="2026-07-13 17:46:33.730900216 +0000 UTC m=+2490.128558595" Jul 13 17:46:43.460546 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:46:43.460512 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-31a784/auth-test-model-31a784-kserve-795957db7-gcbh8" Jul 13 17:46:43.461122 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:46:43.460587 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-31a784/auth-test-model-31a784-kserve-795957db7-gcbh8" Jul 13 17:46:43.472888 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:46:43.472866 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-31a784/auth-test-model-31a784-kserve-795957db7-gcbh8" Jul 13 17:46:43.770983 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:46:43.770901 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-31a784/auth-test-model-31a784-kserve-795957db7-gcbh8" Jul 13 17:48:41.977823 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:41.977741 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-714452/auth-test-model-714452-kserve-7fbf74694c-j4q8q"] Jul 13 17:48:42.966541 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:42.966508 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-31a784/auth-test-model-31a784-kserve-795957db7-gcbh8"] Jul 13 17:48:56.997898 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:56.997835 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-714452/auth-test-model-714452-kserve-7fbf74694c-j4q8q" podUID="3ab939d8-d4d3-4b1f-bb02-8c8670c63c03" containerName="main" containerID="cri-o://38d6648d5a491a9bec3e90b22392cfd18bc48719e86b9d7e0dc801f1799123e2" gracePeriod=45 Jul 13 17:48:57.260805 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:57.260753 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-714452/auth-test-model-714452-kserve-7fbf74694c-j4q8q" Jul 13 17:48:57.306717 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:57.306685 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ab939d8-d4d3-4b1f-bb02-8c8670c63c03-dshm\") pod \"3ab939d8-d4d3-4b1f-bb02-8c8670c63c03\" (UID: \"3ab939d8-d4d3-4b1f-bb02-8c8670c63c03\") " Jul 13 17:48:57.306861 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:57.306727 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ab939d8-d4d3-4b1f-bb02-8c8670c63c03-home\") pod \"3ab939d8-d4d3-4b1f-bb02-8c8670c63c03\" (UID: \"3ab939d8-d4d3-4b1f-bb02-8c8670c63c03\") " Jul 13 17:48:57.306861 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:57.306771 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ab939d8-d4d3-4b1f-bb02-8c8670c63c03-tmp-dir\") pod \"3ab939d8-d4d3-4b1f-bb02-8c8670c63c03\" (UID: \"3ab939d8-d4d3-4b1f-bb02-8c8670c63c03\") " Jul 13 17:48:57.306861 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:57.306794 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tlsgr\" (UniqueName: \"kubernetes.io/projected/3ab939d8-d4d3-4b1f-bb02-8c8670c63c03-kube-api-access-tlsgr\") pod \"3ab939d8-d4d3-4b1f-bb02-8c8670c63c03\" (UID: \"3ab939d8-d4d3-4b1f-bb02-8c8670c63c03\") " Jul 13 17:48:57.306861 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:57.306825 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ab939d8-d4d3-4b1f-bb02-8c8670c63c03-model-cache\") pod \"3ab939d8-d4d3-4b1f-bb02-8c8670c63c03\" (UID: \"3ab939d8-d4d3-4b1f-bb02-8c8670c63c03\") " Jul 13 17:48:57.307079 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:57.306864 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ab939d8-d4d3-4b1f-bb02-8c8670c63c03-tls-certs\") pod \"3ab939d8-d4d3-4b1f-bb02-8c8670c63c03\" (UID: \"3ab939d8-d4d3-4b1f-bb02-8c8670c63c03\") " Jul 13 17:48:57.307079 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:57.307036 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ab939d8-d4d3-4b1f-bb02-8c8670c63c03-home" (OuterVolumeSpecName: "home") pod "3ab939d8-d4d3-4b1f-bb02-8c8670c63c03" (UID: "3ab939d8-d4d3-4b1f-bb02-8c8670c63c03"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:48:57.307079 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:57.307050 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ab939d8-d4d3-4b1f-bb02-8c8670c63c03-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3ab939d8-d4d3-4b1f-bb02-8c8670c63c03" (UID: "3ab939d8-d4d3-4b1f-bb02-8c8670c63c03"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:48:57.307240 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:57.307192 2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ab939d8-d4d3-4b1f-bb02-8c8670c63c03-tmp-dir\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:48:57.307240 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:57.307213 2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ab939d8-d4d3-4b1f-bb02-8c8670c63c03-home\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:48:57.307344 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:57.307225 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ab939d8-d4d3-4b1f-bb02-8c8670c63c03-model-cache" (OuterVolumeSpecName: "model-cache") pod "3ab939d8-d4d3-4b1f-bb02-8c8670c63c03" (UID: "3ab939d8-d4d3-4b1f-bb02-8c8670c63c03"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:48:57.309002 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:57.308979 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ab939d8-d4d3-4b1f-bb02-8c8670c63c03-dshm" (OuterVolumeSpecName: "dshm") pod "3ab939d8-d4d3-4b1f-bb02-8c8670c63c03" (UID: "3ab939d8-d4d3-4b1f-bb02-8c8670c63c03"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:48:57.309002 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:57.308987 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ab939d8-d4d3-4b1f-bb02-8c8670c63c03-kube-api-access-tlsgr" (OuterVolumeSpecName: "kube-api-access-tlsgr") pod "3ab939d8-d4d3-4b1f-bb02-8c8670c63c03" (UID: "3ab939d8-d4d3-4b1f-bb02-8c8670c63c03"). InnerVolumeSpecName "kube-api-access-tlsgr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:48:57.309229 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:57.309062 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ab939d8-d4d3-4b1f-bb02-8c8670c63c03-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3ab939d8-d4d3-4b1f-bb02-8c8670c63c03" (UID: "3ab939d8-d4d3-4b1f-bb02-8c8670c63c03"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 17:48:57.407980 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:57.407947 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tlsgr\" (UniqueName: \"kubernetes.io/projected/3ab939d8-d4d3-4b1f-bb02-8c8670c63c03-kube-api-access-tlsgr\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:48:57.407980 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:57.407975 2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ab939d8-d4d3-4b1f-bb02-8c8670c63c03-model-cache\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:48:57.407980 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:57.407985 2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ab939d8-d4d3-4b1f-bb02-8c8670c63c03-tls-certs\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:48:57.408185 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:57.407993 2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ab939d8-d4d3-4b1f-bb02-8c8670c63c03-dshm\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:48:57.415477 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:57.415445 2569 generic.go:356] "Generic (PLEG): container finished" podID="3ab939d8-d4d3-4b1f-bb02-8c8670c63c03" containerID="38d6648d5a491a9bec3e90b22392cfd18bc48719e86b9d7e0dc801f1799123e2" exitCode=0 Jul 13 17:48:57.415633 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:57.415483 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-714452/auth-test-model-714452-kserve-7fbf74694c-j4q8q" Jul 13 17:48:57.415633 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:57.415531 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-714452/auth-test-model-714452-kserve-7fbf74694c-j4q8q" event={"ID":"3ab939d8-d4d3-4b1f-bb02-8c8670c63c03","Type":"ContainerDied","Data":"38d6648d5a491a9bec3e90b22392cfd18bc48719e86b9d7e0dc801f1799123e2"} Jul 13 17:48:57.415633 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:57.415595 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-714452/auth-test-model-714452-kserve-7fbf74694c-j4q8q" event={"ID":"3ab939d8-d4d3-4b1f-bb02-8c8670c63c03","Type":"ContainerDied","Data":"9e8d125878230c81e04c6b2b57d525d94d3de573131d39abedae80517c9e218f"} Jul 13 17:48:57.415633 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:57.415615 2569 scope.go:117] "RemoveContainer" containerID="38d6648d5a491a9bec3e90b22392cfd18bc48719e86b9d7e0dc801f1799123e2" Jul 13 17:48:57.425136 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:57.425118 2569 scope.go:117] "RemoveContainer" containerID="38d6648d5a491a9bec3e90b22392cfd18bc48719e86b9d7e0dc801f1799123e2" Jul 13 17:48:57.425388 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:48:57.425368 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38d6648d5a491a9bec3e90b22392cfd18bc48719e86b9d7e0dc801f1799123e2\": container with ID starting with 38d6648d5a491a9bec3e90b22392cfd18bc48719e86b9d7e0dc801f1799123e2 not found: ID does not exist" containerID="38d6648d5a491a9bec3e90b22392cfd18bc48719e86b9d7e0dc801f1799123e2" Jul 13 17:48:57.425450 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:57.425402 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38d6648d5a491a9bec3e90b22392cfd18bc48719e86b9d7e0dc801f1799123e2"} err="failed to get container status \"38d6648d5a491a9bec3e90b22392cfd18bc48719e86b9d7e0dc801f1799123e2\": rpc error: code = NotFound desc = could not find container \"38d6648d5a491a9bec3e90b22392cfd18bc48719e86b9d7e0dc801f1799123e2\": container with ID starting with 38d6648d5a491a9bec3e90b22392cfd18bc48719e86b9d7e0dc801f1799123e2 not found: ID does not exist" Jul 13 17:48:57.437414 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:57.437389 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-714452/auth-test-model-714452-kserve-7fbf74694c-j4q8q"] Jul 13 17:48:57.443900 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:57.443879 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-714452/auth-test-model-714452-kserve-7fbf74694c-j4q8q"] Jul 13 17:48:57.987128 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:57.987087 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-31a784/auth-test-model-31a784-kserve-795957db7-gcbh8" podUID="ef047c6e-dd16-4a46-bd3e-54ed872ab4c3" containerName="main" containerID="cri-o://085366bf27a8902953becb168737a68048ed7aa1c85ead7cf0046aa11e8e3532" gracePeriod=45 Jul 13 17:48:58.138756 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:58.138724 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ab939d8-d4d3-4b1f-bb02-8c8670c63c03" path="/var/lib/kubelet/pods/3ab939d8-d4d3-4b1f-bb02-8c8670c63c03/volumes" Jul 13 17:48:58.235710 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:58.235687 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-31a784/auth-test-model-31a784-kserve-795957db7-gcbh8" Jul 13 17:48:58.313828 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:58.313798 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef047c6e-dd16-4a46-bd3e-54ed872ab4c3-home\") pod \"ef047c6e-dd16-4a46-bd3e-54ed872ab4c3\" (UID: \"ef047c6e-dd16-4a46-bd3e-54ed872ab4c3\") " Jul 13 17:48:58.313958 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:58.313851 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef047c6e-dd16-4a46-bd3e-54ed872ab4c3-model-cache\") pod \"ef047c6e-dd16-4a46-bd3e-54ed872ab4c3\" (UID: \"ef047c6e-dd16-4a46-bd3e-54ed872ab4c3\") " Jul 13 17:48:58.313958 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:58.313925 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wgjj7\" (UniqueName: \"kubernetes.io/projected/ef047c6e-dd16-4a46-bd3e-54ed872ab4c3-kube-api-access-wgjj7\") pod \"ef047c6e-dd16-4a46-bd3e-54ed872ab4c3\" (UID: \"ef047c6e-dd16-4a46-bd3e-54ed872ab4c3\") " Jul 13 17:48:58.314068 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:58.313955 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef047c6e-dd16-4a46-bd3e-54ed872ab4c3-tls-certs\") pod \"ef047c6e-dd16-4a46-bd3e-54ed872ab4c3\" (UID: \"ef047c6e-dd16-4a46-bd3e-54ed872ab4c3\") " Jul 13 17:48:58.314068 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:58.313981 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef047c6e-dd16-4a46-bd3e-54ed872ab4c3-tmp-dir\") pod \"ef047c6e-dd16-4a46-bd3e-54ed872ab4c3\" (UID: \"ef047c6e-dd16-4a46-bd3e-54ed872ab4c3\") " Jul 13 17:48:58.314068 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:58.314016 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef047c6e-dd16-4a46-bd3e-54ed872ab4c3-home" (OuterVolumeSpecName: "home") pod "ef047c6e-dd16-4a46-bd3e-54ed872ab4c3" (UID: "ef047c6e-dd16-4a46-bd3e-54ed872ab4c3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:48:58.314068 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:58.314029 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef047c6e-dd16-4a46-bd3e-54ed872ab4c3-dshm\") pod \"ef047c6e-dd16-4a46-bd3e-54ed872ab4c3\" (UID: \"ef047c6e-dd16-4a46-bd3e-54ed872ab4c3\") " Jul 13 17:48:58.314278 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:58.314079 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef047c6e-dd16-4a46-bd3e-54ed872ab4c3-model-cache" (OuterVolumeSpecName: "model-cache") pod "ef047c6e-dd16-4a46-bd3e-54ed872ab4c3" (UID: "ef047c6e-dd16-4a46-bd3e-54ed872ab4c3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:48:58.314336 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:58.314282 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef047c6e-dd16-4a46-bd3e-54ed872ab4c3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ef047c6e-dd16-4a46-bd3e-54ed872ab4c3" (UID: "ef047c6e-dd16-4a46-bd3e-54ed872ab4c3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:48:58.314336 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:58.314326 2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef047c6e-dd16-4a46-bd3e-54ed872ab4c3-home\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:48:58.314401 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:58.314346 2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef047c6e-dd16-4a46-bd3e-54ed872ab4c3-model-cache\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:48:58.316205 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:58.316177 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef047c6e-dd16-4a46-bd3e-54ed872ab4c3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ef047c6e-dd16-4a46-bd3e-54ed872ab4c3" (UID: "ef047c6e-dd16-4a46-bd3e-54ed872ab4c3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 17:48:58.316298 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:58.316209 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef047c6e-dd16-4a46-bd3e-54ed872ab4c3-dshm" (OuterVolumeSpecName: "dshm") pod "ef047c6e-dd16-4a46-bd3e-54ed872ab4c3" (UID: "ef047c6e-dd16-4a46-bd3e-54ed872ab4c3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:48:58.316298 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:58.316252 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef047c6e-dd16-4a46-bd3e-54ed872ab4c3-kube-api-access-wgjj7" (OuterVolumeSpecName: "kube-api-access-wgjj7") pod "ef047c6e-dd16-4a46-bd3e-54ed872ab4c3" (UID: "ef047c6e-dd16-4a46-bd3e-54ed872ab4c3"). InnerVolumeSpecName "kube-api-access-wgjj7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:48:58.414955 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:58.414926 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wgjj7\" (UniqueName: \"kubernetes.io/projected/ef047c6e-dd16-4a46-bd3e-54ed872ab4c3-kube-api-access-wgjj7\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:48:58.414955 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:58.414950 2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef047c6e-dd16-4a46-bd3e-54ed872ab4c3-tls-certs\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:48:58.414955 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:58.414960 2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef047c6e-dd16-4a46-bd3e-54ed872ab4c3-tmp-dir\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:48:58.415168 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:58.414969 2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef047c6e-dd16-4a46-bd3e-54ed872ab4c3-dshm\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:48:58.422804 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:58.422779 2569 generic.go:356] "Generic (PLEG): container finished" podID="ef047c6e-dd16-4a46-bd3e-54ed872ab4c3" containerID="085366bf27a8902953becb168737a68048ed7aa1c85ead7cf0046aa11e8e3532" exitCode=0 Jul 13 17:48:58.422927 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:58.422831 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-31a784/auth-test-model-31a784-kserve-795957db7-gcbh8" Jul 13 17:48:58.422927 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:58.422862 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-31a784/auth-test-model-31a784-kserve-795957db7-gcbh8" event={"ID":"ef047c6e-dd16-4a46-bd3e-54ed872ab4c3","Type":"ContainerDied","Data":"085366bf27a8902953becb168737a68048ed7aa1c85ead7cf0046aa11e8e3532"} Jul 13 17:48:58.422927 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:58.422900 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-31a784/auth-test-model-31a784-kserve-795957db7-gcbh8" event={"ID":"ef047c6e-dd16-4a46-bd3e-54ed872ab4c3","Type":"ContainerDied","Data":"e371a826839f77ea8e3b19d4b94f480060dc90b98dadcacb84d177d26f5a0e4d"} Jul 13 17:48:58.422927 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:58.422919 2569 scope.go:117] "RemoveContainer" containerID="085366bf27a8902953becb168737a68048ed7aa1c85ead7cf0046aa11e8e3532" Jul 13 17:48:58.431881 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:58.431859 2569 scope.go:117] "RemoveContainer" containerID="085366bf27a8902953becb168737a68048ed7aa1c85ead7cf0046aa11e8e3532" Jul 13 17:48:58.432116 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:48:58.432097 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"085366bf27a8902953becb168737a68048ed7aa1c85ead7cf0046aa11e8e3532\": container with ID starting with 085366bf27a8902953becb168737a68048ed7aa1c85ead7cf0046aa11e8e3532 not found: ID does not exist" containerID="085366bf27a8902953becb168737a68048ed7aa1c85ead7cf0046aa11e8e3532" Jul 13 17:48:58.432172 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:58.432124 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"085366bf27a8902953becb168737a68048ed7aa1c85ead7cf0046aa11e8e3532"} err="failed to get container status \"085366bf27a8902953becb168737a68048ed7aa1c85ead7cf0046aa11e8e3532\": rpc error: code = NotFound desc = could not find container \"085366bf27a8902953becb168737a68048ed7aa1c85ead7cf0046aa11e8e3532\": container with ID starting with 085366bf27a8902953becb168737a68048ed7aa1c85ead7cf0046aa11e8e3532 not found: ID does not exist" Jul 13 17:48:58.443825 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:58.443802 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-31a784/auth-test-model-31a784-kserve-795957db7-gcbh8"] Jul 13 17:48:58.448183 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:48:58.448163 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-31a784/auth-test-model-31a784-kserve-795957db7-gcbh8"] Jul 13 17:49:00.137301 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:49:00.137264 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef047c6e-dd16-4a46-bd3e-54ed872ab4c3" path="/var/lib/kubelet/pods/ef047c6e-dd16-4a46-bd3e-54ed872ab4c3/volumes" Jul 13 17:49:18.042932 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:49:18.042901 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-31a784/sub-test-model-31a784-kserve-d644b99cf-l2tfs"] Jul 13 17:49:18.043296 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:49:18.043239 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef047c6e-dd16-4a46-bd3e-54ed872ab4c3" containerName="main" Jul 13 17:49:18.043296 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:49:18.043250 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef047c6e-dd16-4a46-bd3e-54ed872ab4c3" containerName="main" Jul 13 17:49:18.043296 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:49:18.043270 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ab939d8-d4d3-4b1f-bb02-8c8670c63c03" containerName="main" Jul 13 17:49:18.043296 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:49:18.043275 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ab939d8-d4d3-4b1f-bb02-8c8670c63c03" containerName="main" Jul 13 17:49:18.043446 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:49:18.043338 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef047c6e-dd16-4a46-bd3e-54ed872ab4c3" containerName="main" Jul 13 17:49:18.043446 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:49:18.043350 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ab939d8-d4d3-4b1f-bb02-8c8670c63c03" containerName="main" Jul 13 17:49:18.046309 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:49:18.046294 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-31a784/sub-test-model-31a784-kserve-d644b99cf-l2tfs" Jul 13 17:49:18.049743 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:49:18.049721 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-31a784\"/\"kube-root-ca.crt\"" Jul 13 17:49:18.049743 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:49:18.049735 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-31a784\"/\"sub-test-model-31a784-kserve-self-signed-certs\"" Jul 13 17:49:18.049916 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:49:18.049723 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-31a784\"/\"default-dockercfg-d2rx8\"" Jul 13 17:49:18.050125 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:49:18.050109 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-31a784\"/\"openshift-service-ca.crt\"" Jul 13 17:49:18.068106 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:49:18.068086 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-31a784/sub-test-model-31a784-kserve-d644b99cf-l2tfs"] Jul 13 17:49:18.164728 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:49:18.164697 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0749e5ff-a060-4c1a-ac8e-97d247fd0770-model-cache\") pod \"sub-test-model-31a784-kserve-d644b99cf-l2tfs\" (UID: \"0749e5ff-a060-4c1a-ac8e-97d247fd0770\") " pod="ai-tenant-e2e-shared-b-31a784/sub-test-model-31a784-kserve-d644b99cf-l2tfs" Jul 13 17:49:18.164853 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:49:18.164753 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0749e5ff-a060-4c1a-ac8e-97d247fd0770-dshm\") pod \"sub-test-model-31a784-kserve-d644b99cf-l2tfs\" (UID: \"0749e5ff-a060-4c1a-ac8e-97d247fd0770\") " pod="ai-tenant-e2e-shared-b-31a784/sub-test-model-31a784-kserve-d644b99cf-l2tfs" Jul 13 17:49:18.164853 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:49:18.164793 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0749e5ff-a060-4c1a-ac8e-97d247fd0770-home\") pod \"sub-test-model-31a784-kserve-d644b99cf-l2tfs\" (UID: \"0749e5ff-a060-4c1a-ac8e-97d247fd0770\") " pod="ai-tenant-e2e-shared-b-31a784/sub-test-model-31a784-kserve-d644b99cf-l2tfs" Jul 13 17:49:18.164853 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:49:18.164816 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0749e5ff-a060-4c1a-ac8e-97d247fd0770-tmp-dir\") pod \"sub-test-model-31a784-kserve-d644b99cf-l2tfs\" (UID: \"0749e5ff-a060-4c1a-ac8e-97d247fd0770\") " pod="ai-tenant-e2e-shared-b-31a784/sub-test-model-31a784-kserve-d644b99cf-l2tfs" Jul 13 17:49:18.164853 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:49:18.164841 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0749e5ff-a060-4c1a-ac8e-97d247fd0770-tls-certs\") pod \"sub-test-model-31a784-kserve-d644b99cf-l2tfs\" (UID: \"0749e5ff-a060-4c1a-ac8e-97d247fd0770\") " pod="ai-tenant-e2e-shared-b-31a784/sub-test-model-31a784-kserve-d644b99cf-l2tfs" Jul 13 17:49:18.165010 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:49:18.164898 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhhkk\" (UniqueName: \"kubernetes.io/projected/0749e5ff-a060-4c1a-ac8e-97d247fd0770-kube-api-access-qhhkk\") pod \"sub-test-model-31a784-kserve-d644b99cf-l2tfs\" (UID: \"0749e5ff-a060-4c1a-ac8e-97d247fd0770\") " pod="ai-tenant-e2e-shared-b-31a784/sub-test-model-31a784-kserve-d644b99cf-l2tfs" Jul 13 17:49:18.265878 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:49:18.265849 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0749e5ff-a060-4c1a-ac8e-97d247fd0770-model-cache\") pod \"sub-test-model-31a784-kserve-d644b99cf-l2tfs\" (UID: \"0749e5ff-a060-4c1a-ac8e-97d247fd0770\") " pod="ai-tenant-e2e-shared-b-31a784/sub-test-model-31a784-kserve-d644b99cf-l2tfs" Jul 13 17:49:18.266016 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:49:18.265899 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0749e5ff-a060-4c1a-ac8e-97d247fd0770-dshm\") pod \"sub-test-model-31a784-kserve-d644b99cf-l2tfs\" (UID: \"0749e5ff-a060-4c1a-ac8e-97d247fd0770\") " pod="ai-tenant-e2e-shared-b-31a784/sub-test-model-31a784-kserve-d644b99cf-l2tfs" Jul 13 17:49:18.266016 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:49:18.265919 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0749e5ff-a060-4c1a-ac8e-97d247fd0770-home\") pod \"sub-test-model-31a784-kserve-d644b99cf-l2tfs\" (UID: \"0749e5ff-a060-4c1a-ac8e-97d247fd0770\") " pod="ai-tenant-e2e-shared-b-31a784/sub-test-model-31a784-kserve-d644b99cf-l2tfs" Jul 13 17:49:18.266016 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:49:18.265941 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0749e5ff-a060-4c1a-ac8e-97d247fd0770-tmp-dir\") pod \"sub-test-model-31a784-kserve-d644b99cf-l2tfs\" (UID: \"0749e5ff-a060-4c1a-ac8e-97d247fd0770\") " pod="ai-tenant-e2e-shared-b-31a784/sub-test-model-31a784-kserve-d644b99cf-l2tfs" Jul 13 17:49:18.266016 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:49:18.265958 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0749e5ff-a060-4c1a-ac8e-97d247fd0770-tls-certs\") pod \"sub-test-model-31a784-kserve-d644b99cf-l2tfs\" (UID: \"0749e5ff-a060-4c1a-ac8e-97d247fd0770\") " pod="ai-tenant-e2e-shared-b-31a784/sub-test-model-31a784-kserve-d644b99cf-l2tfs" Jul 13 17:49:18.266016 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:49:18.265977 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhhkk\" (UniqueName: \"kubernetes.io/projected/0749e5ff-a060-4c1a-ac8e-97d247fd0770-kube-api-access-qhhkk\") pod \"sub-test-model-31a784-kserve-d644b99cf-l2tfs\" (UID: \"0749e5ff-a060-4c1a-ac8e-97d247fd0770\") " pod="ai-tenant-e2e-shared-b-31a784/sub-test-model-31a784-kserve-d644b99cf-l2tfs" Jul 13 17:49:18.266281 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:49:18.266260 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0749e5ff-a060-4c1a-ac8e-97d247fd0770-model-cache\") pod \"sub-test-model-31a784-kserve-d644b99cf-l2tfs\" (UID: \"0749e5ff-a060-4c1a-ac8e-97d247fd0770\") " pod="ai-tenant-e2e-shared-b-31a784/sub-test-model-31a784-kserve-d644b99cf-l2tfs" Jul 13 17:49:18.266331 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:49:18.266304 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0749e5ff-a060-4c1a-ac8e-97d247fd0770-home\") pod \"sub-test-model-31a784-kserve-d644b99cf-l2tfs\" (UID: \"0749e5ff-a060-4c1a-ac8e-97d247fd0770\") " pod="ai-tenant-e2e-shared-b-31a784/sub-test-model-31a784-kserve-d644b99cf-l2tfs" Jul 13 17:49:18.266365 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:49:18.266325 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0749e5ff-a060-4c1a-ac8e-97d247fd0770-tmp-dir\") pod \"sub-test-model-31a784-kserve-d644b99cf-l2tfs\" (UID: \"0749e5ff-a060-4c1a-ac8e-97d247fd0770\") " pod="ai-tenant-e2e-shared-b-31a784/sub-test-model-31a784-kserve-d644b99cf-l2tfs" Jul 13 17:49:18.268254 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:49:18.268232 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0749e5ff-a060-4c1a-ac8e-97d247fd0770-dshm\") pod \"sub-test-model-31a784-kserve-d644b99cf-l2tfs\" (UID: \"0749e5ff-a060-4c1a-ac8e-97d247fd0770\") " pod="ai-tenant-e2e-shared-b-31a784/sub-test-model-31a784-kserve-d644b99cf-l2tfs" Jul 13 17:49:18.268397 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:49:18.268378 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0749e5ff-a060-4c1a-ac8e-97d247fd0770-tls-certs\") pod \"sub-test-model-31a784-kserve-d644b99cf-l2tfs\" (UID: \"0749e5ff-a060-4c1a-ac8e-97d247fd0770\") " pod="ai-tenant-e2e-shared-b-31a784/sub-test-model-31a784-kserve-d644b99cf-l2tfs" Jul 13 17:49:18.275904 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:49:18.275872 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhhkk\" (UniqueName: \"kubernetes.io/projected/0749e5ff-a060-4c1a-ac8e-97d247fd0770-kube-api-access-qhhkk\") pod \"sub-test-model-31a784-kserve-d644b99cf-l2tfs\" (UID: \"0749e5ff-a060-4c1a-ac8e-97d247fd0770\") " pod="ai-tenant-e2e-shared-b-31a784/sub-test-model-31a784-kserve-d644b99cf-l2tfs" Jul 13 17:49:18.355930 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:49:18.355857 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-31a784/sub-test-model-31a784-kserve-d644b99cf-l2tfs" Jul 13 17:49:18.502624 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:49:18.502592 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-31a784/sub-test-model-31a784-kserve-d644b99cf-l2tfs"] Jul 13 17:49:18.508699 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:49:18.508671 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0749e5ff_a060_4c1a_ac8e_97d247fd0770.slice/crio-912e73bba701c166034a1192cb6b338f1568ee50deb469a5c6b771d605f13c63 WatchSource:0}: Error finding container 912e73bba701c166034a1192cb6b338f1568ee50deb469a5c6b771d605f13c63: Status 404 returned error can't find the container with id 912e73bba701c166034a1192cb6b338f1568ee50deb469a5c6b771d605f13c63 Jul 13 17:49:18.523096 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:49:18.523060 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-31a784/sub-test-model-31a784-kserve-d644b99cf-l2tfs" event={"ID":"0749e5ff-a060-4c1a-ac8e-97d247fd0770","Type":"ContainerStarted","Data":"912e73bba701c166034a1192cb6b338f1568ee50deb469a5c6b771d605f13c63"} Jul 13 17:49:19.529676 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:49:19.529636 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-31a784/sub-test-model-31a784-kserve-d644b99cf-l2tfs" event={"ID":"0749e5ff-a060-4c1a-ac8e-97d247fd0770","Type":"ContainerStarted","Data":"182945485f648835c40b7de6d075d17094142d5d3f7ab66b1920ff701c8b27f1"} Jul 13 17:49:19.550019 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:49:19.549973 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-31a784/sub-test-model-31a784-kserve-d644b99cf-l2tfs" podStartSLOduration=1.5499585489999999 podStartE2EDuration="1.549958549s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:49:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:49:19.549620792 +0000 UTC m=+2655.947279172" watchObservedRunningTime="2026-07-13 17:49:19.549958549 +0000 UTC m=+2655.947616928" Jul 13 17:49:28.356185 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:49:28.356141 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-31a784/sub-test-model-31a784-kserve-d644b99cf-l2tfs" Jul 13 17:49:28.356699 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:49:28.356219 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-31a784/sub-test-model-31a784-kserve-d644b99cf-l2tfs" Jul 13 17:49:28.368899 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:49:28.368870 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-31a784/sub-test-model-31a784-kserve-d644b99cf-l2tfs" Jul 13 17:49:28.584508 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:49:28.584473 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-31a784/sub-test-model-31a784-kserve-d644b99cf-l2tfs" Jul 13 17:50:04.274147 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:04.274058 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-hzcql_c40e6e88-ab3f-4e3c-a0e7-2400bda333eb/console-operator/1.log" Jul 13 17:50:04.282750 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:04.282728 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q2pnb_6a531b70-4a58-4752-8fa8-5ba634b542ea/ovn-acl-logging/0.log" Jul 13 17:50:04.285826 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:04.285808 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-hzcql_c40e6e88-ab3f-4e3c-a0e7-2400bda333eb/console-operator/1.log" Jul 13 17:50:04.294123 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:04.294106 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q2pnb_6a531b70-4a58-4752-8fa8-5ba634b542ea/ovn-acl-logging/0.log" Jul 13 17:50:17.550644 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:17.550611 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-31a784/sub-test-model-31a784-kserve-d644b99cf-l2tfs"] Jul 13 17:50:22.237349 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:22.237314 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-714452/rate-test-model-714452-kserve-54b459cbcf-wx8kj"] Jul 13 17:50:22.240681 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:22.240665 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-714452/rate-test-model-714452-kserve-54b459cbcf-wx8kj" Jul 13 17:50:22.243835 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:22.243802 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-714452\"/\"rate-test-model-714452-kserve-self-signed-certs\"" Jul 13 17:50:22.243835 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:22.243833 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-714452\"/\"kube-root-ca.crt\"" Jul 13 17:50:22.244026 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:22.243830 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-714452\"/\"default-dockercfg-s6hqf\"" Jul 13 17:50:22.244026 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:22.243810 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-714452\"/\"openshift-service-ca.crt\"" Jul 13 17:50:22.255236 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:22.255209 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-714452/rate-test-model-714452-kserve-54b459cbcf-wx8kj"] Jul 13 17:50:22.305402 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:22.305377 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e13ce04-baf2-4118-8fab-7518c82b5dd6-home\") pod \"rate-test-model-714452-kserve-54b459cbcf-wx8kj\" (UID: \"5e13ce04-baf2-4118-8fab-7518c82b5dd6\") " pod="ai-tenant-e2e-shared-a-714452/rate-test-model-714452-kserve-54b459cbcf-wx8kj" Jul 13 17:50:22.305522 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:22.305411 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e13ce04-baf2-4118-8fab-7518c82b5dd6-tls-certs\") pod \"rate-test-model-714452-kserve-54b459cbcf-wx8kj\" (UID: \"5e13ce04-baf2-4118-8fab-7518c82b5dd6\") " pod="ai-tenant-e2e-shared-a-714452/rate-test-model-714452-kserve-54b459cbcf-wx8kj" Jul 13 17:50:22.305522 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:22.305430 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e13ce04-baf2-4118-8fab-7518c82b5dd6-dshm\") pod \"rate-test-model-714452-kserve-54b459cbcf-wx8kj\" (UID: \"5e13ce04-baf2-4118-8fab-7518c82b5dd6\") " pod="ai-tenant-e2e-shared-a-714452/rate-test-model-714452-kserve-54b459cbcf-wx8kj" Jul 13 17:50:22.305522 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:22.305494 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcjm6\" (UniqueName: \"kubernetes.io/projected/5e13ce04-baf2-4118-8fab-7518c82b5dd6-kube-api-access-dcjm6\") pod \"rate-test-model-714452-kserve-54b459cbcf-wx8kj\" (UID: \"5e13ce04-baf2-4118-8fab-7518c82b5dd6\") " pod="ai-tenant-e2e-shared-a-714452/rate-test-model-714452-kserve-54b459cbcf-wx8kj" Jul 13 17:50:22.305688 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:22.305550 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e13ce04-baf2-4118-8fab-7518c82b5dd6-model-cache\") pod \"rate-test-model-714452-kserve-54b459cbcf-wx8kj\" (UID: \"5e13ce04-baf2-4118-8fab-7518c82b5dd6\") " pod="ai-tenant-e2e-shared-a-714452/rate-test-model-714452-kserve-54b459cbcf-wx8kj" Jul 13 17:50:22.305688 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:22.305626 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e13ce04-baf2-4118-8fab-7518c82b5dd6-tmp-dir\") pod \"rate-test-model-714452-kserve-54b459cbcf-wx8kj\" (UID: \"5e13ce04-baf2-4118-8fab-7518c82b5dd6\") " pod="ai-tenant-e2e-shared-a-714452/rate-test-model-714452-kserve-54b459cbcf-wx8kj" Jul 13 17:50:22.406747 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:22.406715 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcjm6\" (UniqueName: \"kubernetes.io/projected/5e13ce04-baf2-4118-8fab-7518c82b5dd6-kube-api-access-dcjm6\") pod \"rate-test-model-714452-kserve-54b459cbcf-wx8kj\" (UID: \"5e13ce04-baf2-4118-8fab-7518c82b5dd6\") " pod="ai-tenant-e2e-shared-a-714452/rate-test-model-714452-kserve-54b459cbcf-wx8kj" Jul 13 17:50:22.406919 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:22.406766 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e13ce04-baf2-4118-8fab-7518c82b5dd6-model-cache\") pod \"rate-test-model-714452-kserve-54b459cbcf-wx8kj\" (UID: \"5e13ce04-baf2-4118-8fab-7518c82b5dd6\") " pod="ai-tenant-e2e-shared-a-714452/rate-test-model-714452-kserve-54b459cbcf-wx8kj" Jul 13 17:50:22.406919 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:22.406814 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e13ce04-baf2-4118-8fab-7518c82b5dd6-tmp-dir\") pod \"rate-test-model-714452-kserve-54b459cbcf-wx8kj\" (UID: \"5e13ce04-baf2-4118-8fab-7518c82b5dd6\") " pod="ai-tenant-e2e-shared-a-714452/rate-test-model-714452-kserve-54b459cbcf-wx8kj" Jul 13 17:50:22.406919 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:22.406861 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e13ce04-baf2-4118-8fab-7518c82b5dd6-home\") pod \"rate-test-model-714452-kserve-54b459cbcf-wx8kj\" (UID: \"5e13ce04-baf2-4118-8fab-7518c82b5dd6\") " pod="ai-tenant-e2e-shared-a-714452/rate-test-model-714452-kserve-54b459cbcf-wx8kj" Jul 13 17:50:22.407080 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:22.406980 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e13ce04-baf2-4118-8fab-7518c82b5dd6-tls-certs\") pod \"rate-test-model-714452-kserve-54b459cbcf-wx8kj\" (UID: \"5e13ce04-baf2-4118-8fab-7518c82b5dd6\") " pod="ai-tenant-e2e-shared-a-714452/rate-test-model-714452-kserve-54b459cbcf-wx8kj" Jul 13 17:50:22.407080 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:22.407019 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e13ce04-baf2-4118-8fab-7518c82b5dd6-dshm\") pod \"rate-test-model-714452-kserve-54b459cbcf-wx8kj\" (UID: \"5e13ce04-baf2-4118-8fab-7518c82b5dd6\") " pod="ai-tenant-e2e-shared-a-714452/rate-test-model-714452-kserve-54b459cbcf-wx8kj" Jul 13 17:50:22.407294 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:22.407271 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e13ce04-baf2-4118-8fab-7518c82b5dd6-tmp-dir\") pod \"rate-test-model-714452-kserve-54b459cbcf-wx8kj\" (UID: \"5e13ce04-baf2-4118-8fab-7518c82b5dd6\") " pod="ai-tenant-e2e-shared-a-714452/rate-test-model-714452-kserve-54b459cbcf-wx8kj" Jul 13 17:50:22.407360 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:22.407282 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e13ce04-baf2-4118-8fab-7518c82b5dd6-home\") pod \"rate-test-model-714452-kserve-54b459cbcf-wx8kj\" (UID: \"5e13ce04-baf2-4118-8fab-7518c82b5dd6\") " pod="ai-tenant-e2e-shared-a-714452/rate-test-model-714452-kserve-54b459cbcf-wx8kj" Jul 13 17:50:22.407360 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:22.407324 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e13ce04-baf2-4118-8fab-7518c82b5dd6-model-cache\") pod \"rate-test-model-714452-kserve-54b459cbcf-wx8kj\" (UID: \"5e13ce04-baf2-4118-8fab-7518c82b5dd6\") " pod="ai-tenant-e2e-shared-a-714452/rate-test-model-714452-kserve-54b459cbcf-wx8kj" Jul 13 17:50:22.409291 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:22.409271 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e13ce04-baf2-4118-8fab-7518c82b5dd6-dshm\") pod \"rate-test-model-714452-kserve-54b459cbcf-wx8kj\" (UID: \"5e13ce04-baf2-4118-8fab-7518c82b5dd6\") " pod="ai-tenant-e2e-shared-a-714452/rate-test-model-714452-kserve-54b459cbcf-wx8kj" Jul 13 17:50:22.409597 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:22.409577 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e13ce04-baf2-4118-8fab-7518c82b5dd6-tls-certs\") pod \"rate-test-model-714452-kserve-54b459cbcf-wx8kj\" (UID: \"5e13ce04-baf2-4118-8fab-7518c82b5dd6\") " pod="ai-tenant-e2e-shared-a-714452/rate-test-model-714452-kserve-54b459cbcf-wx8kj" Jul 13 17:50:22.415017 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:22.414997 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcjm6\" (UniqueName: \"kubernetes.io/projected/5e13ce04-baf2-4118-8fab-7518c82b5dd6-kube-api-access-dcjm6\") pod \"rate-test-model-714452-kserve-54b459cbcf-wx8kj\" (UID: \"5e13ce04-baf2-4118-8fab-7518c82b5dd6\") " pod="ai-tenant-e2e-shared-a-714452/rate-test-model-714452-kserve-54b459cbcf-wx8kj" Jul 13 17:50:22.558692 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:22.558665 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-714452/rate-test-model-714452-kserve-54b459cbcf-wx8kj" Jul 13 17:50:22.700153 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:22.700126 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-714452/rate-test-model-714452-kserve-54b459cbcf-wx8kj"] Jul 13 17:50:22.702422 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:50:22.702397 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5e13ce04_baf2_4118_8fab_7518c82b5dd6.slice/crio-584d90461e8f3229ece1b3edc1ca9a6dbb906784e95f869ae70fe2e6d3fd98db WatchSource:0}: Error finding container 584d90461e8f3229ece1b3edc1ca9a6dbb906784e95f869ae70fe2e6d3fd98db: Status 404 returned error can't find the container with id 584d90461e8f3229ece1b3edc1ca9a6dbb906784e95f869ae70fe2e6d3fd98db Jul 13 17:50:22.839089 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:22.838997 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-714452/rate-test-model-714452-kserve-54b459cbcf-wx8kj" event={"ID":"5e13ce04-baf2-4118-8fab-7518c82b5dd6","Type":"ContainerStarted","Data":"0ae79beb4c48615ad87191c5ce7eddbd5d0e8b5febc34c7e8524458b91e91548"} Jul 13 17:50:22.839089 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:22.839044 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-714452/rate-test-model-714452-kserve-54b459cbcf-wx8kj" event={"ID":"5e13ce04-baf2-4118-8fab-7518c82b5dd6","Type":"ContainerStarted","Data":"584d90461e8f3229ece1b3edc1ca9a6dbb906784e95f869ae70fe2e6d3fd98db"} Jul 13 17:50:22.862634 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:22.862586 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-714452/rate-test-model-714452-kserve-54b459cbcf-wx8kj" podStartSLOduration=0.862558001 podStartE2EDuration="862.558001ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:50:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:50:22.862306606 +0000 UTC m=+2719.259964986" watchObservedRunningTime="2026-07-13 17:50:22.862558001 +0000 UTC m=+2719.260216407" Jul 13 17:50:32.558863 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:32.558830 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-714452/rate-test-model-714452-kserve-54b459cbcf-wx8kj" Jul 13 17:50:32.559295 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:32.558882 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-714452/rate-test-model-714452-kserve-54b459cbcf-wx8kj" Jul 13 17:50:32.570272 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:32.570234 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-31a784/sub-test-model-31a784-kserve-d644b99cf-l2tfs" podUID="0749e5ff-a060-4c1a-ac8e-97d247fd0770" containerName="main" containerID="cri-o://182945485f648835c40b7de6d075d17094142d5d3f7ab66b1920ff701c8b27f1" gracePeriod=45 Jul 13 17:50:32.571621 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:32.571599 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-714452/rate-test-model-714452-kserve-54b459cbcf-wx8kj" Jul 13 17:50:32.823501 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:32.823446 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-31a784/sub-test-model-31a784-kserve-d644b99cf-l2tfs" Jul 13 17:50:32.886148 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:32.886122 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0749e5ff-a060-4c1a-ac8e-97d247fd0770-model-cache\") pod \"0749e5ff-a060-4c1a-ac8e-97d247fd0770\" (UID: \"0749e5ff-a060-4c1a-ac8e-97d247fd0770\") " Jul 13 17:50:32.886277 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:32.886153 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0749e5ff-a060-4c1a-ac8e-97d247fd0770-home\") pod \"0749e5ff-a060-4c1a-ac8e-97d247fd0770\" (UID: \"0749e5ff-a060-4c1a-ac8e-97d247fd0770\") " Jul 13 17:50:32.886277 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:32.886191 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0749e5ff-a060-4c1a-ac8e-97d247fd0770-tls-certs\") pod \"0749e5ff-a060-4c1a-ac8e-97d247fd0770\" (UID: \"0749e5ff-a060-4c1a-ac8e-97d247fd0770\") " Jul 13 17:50:32.886277 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:32.886216 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0749e5ff-a060-4c1a-ac8e-97d247fd0770-tmp-dir\") pod \"0749e5ff-a060-4c1a-ac8e-97d247fd0770\" (UID: \"0749e5ff-a060-4c1a-ac8e-97d247fd0770\") " Jul 13 17:50:32.886277 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:32.886249 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qhhkk\" (UniqueName: \"kubernetes.io/projected/0749e5ff-a060-4c1a-ac8e-97d247fd0770-kube-api-access-qhhkk\") pod \"0749e5ff-a060-4c1a-ac8e-97d247fd0770\" (UID: \"0749e5ff-a060-4c1a-ac8e-97d247fd0770\") " Jul 13 17:50:32.886477 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:32.886417 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0749e5ff-a060-4c1a-ac8e-97d247fd0770-dshm\") pod \"0749e5ff-a060-4c1a-ac8e-97d247fd0770\" (UID: \"0749e5ff-a060-4c1a-ac8e-97d247fd0770\") " Jul 13 17:50:32.886530 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:32.886439 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0749e5ff-a060-4c1a-ac8e-97d247fd0770-model-cache" (OuterVolumeSpecName: "model-cache") pod "0749e5ff-a060-4c1a-ac8e-97d247fd0770" (UID: "0749e5ff-a060-4c1a-ac8e-97d247fd0770"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:50:32.886530 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:32.886447 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0749e5ff-a060-4c1a-ac8e-97d247fd0770-home" (OuterVolumeSpecName: "home") pod "0749e5ff-a060-4c1a-ac8e-97d247fd0770" (UID: "0749e5ff-a060-4c1a-ac8e-97d247fd0770"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:50:32.886530 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:32.886521 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0749e5ff-a060-4c1a-ac8e-97d247fd0770-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0749e5ff-a060-4c1a-ac8e-97d247fd0770" (UID: "0749e5ff-a060-4c1a-ac8e-97d247fd0770"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:50:32.886799 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:32.886780 2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0749e5ff-a060-4c1a-ac8e-97d247fd0770-model-cache\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:50:32.886799 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:32.886803 2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0749e5ff-a060-4c1a-ac8e-97d247fd0770-home\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:50:32.886905 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:32.886816 2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0749e5ff-a060-4c1a-ac8e-97d247fd0770-tmp-dir\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:50:32.888631 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:32.888609 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0749e5ff-a060-4c1a-ac8e-97d247fd0770-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0749e5ff-a060-4c1a-ac8e-97d247fd0770" (UID: "0749e5ff-a060-4c1a-ac8e-97d247fd0770"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 17:50:32.888739 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:32.888723 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0749e5ff-a060-4c1a-ac8e-97d247fd0770-dshm" (OuterVolumeSpecName: "dshm") pod "0749e5ff-a060-4c1a-ac8e-97d247fd0770" (UID: "0749e5ff-a060-4c1a-ac8e-97d247fd0770"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:50:32.888904 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:32.888873 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0749e5ff-a060-4c1a-ac8e-97d247fd0770-kube-api-access-qhhkk" (OuterVolumeSpecName: "kube-api-access-qhhkk") pod "0749e5ff-a060-4c1a-ac8e-97d247fd0770" (UID: "0749e5ff-a060-4c1a-ac8e-97d247fd0770"). InnerVolumeSpecName "kube-api-access-qhhkk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:50:32.891172 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:32.891146 2569 generic.go:356] "Generic (PLEG): container finished" podID="0749e5ff-a060-4c1a-ac8e-97d247fd0770" containerID="182945485f648835c40b7de6d075d17094142d5d3f7ab66b1920ff701c8b27f1" exitCode=0 Jul 13 17:50:32.891279 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:32.891192 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-31a784/sub-test-model-31a784-kserve-d644b99cf-l2tfs" Jul 13 17:50:32.891279 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:32.891229 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-31a784/sub-test-model-31a784-kserve-d644b99cf-l2tfs" event={"ID":"0749e5ff-a060-4c1a-ac8e-97d247fd0770","Type":"ContainerDied","Data":"182945485f648835c40b7de6d075d17094142d5d3f7ab66b1920ff701c8b27f1"} Jul 13 17:50:32.891279 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:32.891275 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-31a784/sub-test-model-31a784-kserve-d644b99cf-l2tfs" event={"ID":"0749e5ff-a060-4c1a-ac8e-97d247fd0770","Type":"ContainerDied","Data":"912e73bba701c166034a1192cb6b338f1568ee50deb469a5c6b771d605f13c63"} Jul 13 17:50:32.891490 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:32.891294 2569 scope.go:117] "RemoveContainer" containerID="182945485f648835c40b7de6d075d17094142d5d3f7ab66b1920ff701c8b27f1" Jul 13 17:50:32.903336 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:32.903319 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-714452/rate-test-model-714452-kserve-54b459cbcf-wx8kj" Jul 13 17:50:32.909603 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:32.909507 2569 scope.go:117] "RemoveContainer" containerID="182945485f648835c40b7de6d075d17094142d5d3f7ab66b1920ff701c8b27f1" Jul 13 17:50:32.909916 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:50:32.909893 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"182945485f648835c40b7de6d075d17094142d5d3f7ab66b1920ff701c8b27f1\": container with ID starting with 182945485f648835c40b7de6d075d17094142d5d3f7ab66b1920ff701c8b27f1 not found: ID does not exist" containerID="182945485f648835c40b7de6d075d17094142d5d3f7ab66b1920ff701c8b27f1" Jul 13 17:50:32.909996 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:32.909927 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"182945485f648835c40b7de6d075d17094142d5d3f7ab66b1920ff701c8b27f1"} err="failed to get container status \"182945485f648835c40b7de6d075d17094142d5d3f7ab66b1920ff701c8b27f1\": rpc error: code = NotFound desc = could not find container \"182945485f648835c40b7de6d075d17094142d5d3f7ab66b1920ff701c8b27f1\": container with ID starting with 182945485f648835c40b7de6d075d17094142d5d3f7ab66b1920ff701c8b27f1 not found: ID does not exist" Jul 13 17:50:32.941811 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:32.941783 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-31a784/sub-test-model-31a784-kserve-d644b99cf-l2tfs"] Jul 13 17:50:32.947428 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:32.947403 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-31a784/sub-test-model-31a784-kserve-d644b99cf-l2tfs"] Jul 13 17:50:32.988142 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:32.988114 2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0749e5ff-a060-4c1a-ac8e-97d247fd0770-dshm\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:50:32.988142 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:32.988140 2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0749e5ff-a060-4c1a-ac8e-97d247fd0770-tls-certs\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:50:32.988327 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:32.988227 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qhhkk\" (UniqueName: \"kubernetes.io/projected/0749e5ff-a060-4c1a-ac8e-97d247fd0770-kube-api-access-qhhkk\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:50:34.137544 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:50:34.137501 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0749e5ff-a060-4c1a-ac8e-97d247fd0770" path="/var/lib/kubelet/pods/0749e5ff-a060-4c1a-ac8e-97d247fd0770/volumes" Jul 13 17:51:04.363651 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:51:04.363623 2569 scope.go:117] "RemoveContainer" containerID="ec854f28e424e2b1308fe3ff078214c6d76e4a261bc8cbc5b92740393740fd1e" Jul 13 17:51:39.020101 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:51:39.020025 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-714452/rate-test-model-714452-kserve-54b459cbcf-wx8kj"] Jul 13 17:51:54.039951 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:51:54.039904 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-714452/rate-test-model-714452-kserve-54b459cbcf-wx8kj" podUID="5e13ce04-baf2-4118-8fab-7518c82b5dd6" containerName="main" containerID="cri-o://0ae79beb4c48615ad87191c5ce7eddbd5d0e8b5febc34c7e8524458b91e91548" gracePeriod=45 Jul 13 17:51:54.291677 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:51:54.291623 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-714452/rate-test-model-714452-kserve-54b459cbcf-wx8kj" Jul 13 17:51:54.305076 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:51:54.305046 2569 generic.go:356] "Generic (PLEG): container finished" podID="5e13ce04-baf2-4118-8fab-7518c82b5dd6" containerID="0ae79beb4c48615ad87191c5ce7eddbd5d0e8b5febc34c7e8524458b91e91548" exitCode=0 Jul 13 17:51:54.305182 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:51:54.305081 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-714452/rate-test-model-714452-kserve-54b459cbcf-wx8kj" Jul 13 17:51:54.305182 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:51:54.305128 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-714452/rate-test-model-714452-kserve-54b459cbcf-wx8kj" event={"ID":"5e13ce04-baf2-4118-8fab-7518c82b5dd6","Type":"ContainerDied","Data":"0ae79beb4c48615ad87191c5ce7eddbd5d0e8b5febc34c7e8524458b91e91548"} Jul 13 17:51:54.305182 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:51:54.305172 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-714452/rate-test-model-714452-kserve-54b459cbcf-wx8kj" event={"ID":"5e13ce04-baf2-4118-8fab-7518c82b5dd6","Type":"ContainerDied","Data":"584d90461e8f3229ece1b3edc1ca9a6dbb906784e95f869ae70fe2e6d3fd98db"} Jul 13 17:51:54.305300 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:51:54.305192 2569 scope.go:117] "RemoveContainer" containerID="0ae79beb4c48615ad87191c5ce7eddbd5d0e8b5febc34c7e8524458b91e91548" Jul 13 17:51:54.314119 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:51:54.314101 2569 scope.go:117] "RemoveContainer" containerID="0ae79beb4c48615ad87191c5ce7eddbd5d0e8b5febc34c7e8524458b91e91548" Jul 13 17:51:54.314440 ip-10-0-136-220 kubenswrapper[2569]: E0713 17:51:54.314417 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ae79beb4c48615ad87191c5ce7eddbd5d0e8b5febc34c7e8524458b91e91548\": container with ID starting with 0ae79beb4c48615ad87191c5ce7eddbd5d0e8b5febc34c7e8524458b91e91548 not found: ID does not exist" containerID="0ae79beb4c48615ad87191c5ce7eddbd5d0e8b5febc34c7e8524458b91e91548" Jul 13 17:51:54.314541 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:51:54.314446 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ae79beb4c48615ad87191c5ce7eddbd5d0e8b5febc34c7e8524458b91e91548"} err="failed to get container status \"0ae79beb4c48615ad87191c5ce7eddbd5d0e8b5febc34c7e8524458b91e91548\": rpc error: code = NotFound desc = could not find container \"0ae79beb4c48615ad87191c5ce7eddbd5d0e8b5febc34c7e8524458b91e91548\": container with ID starting with 0ae79beb4c48615ad87191c5ce7eddbd5d0e8b5febc34c7e8524458b91e91548 not found: ID does not exist" Jul 13 17:51:54.463610 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:51:54.463550 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dcjm6\" (UniqueName: \"kubernetes.io/projected/5e13ce04-baf2-4118-8fab-7518c82b5dd6-kube-api-access-dcjm6\") pod \"5e13ce04-baf2-4118-8fab-7518c82b5dd6\" (UID: \"5e13ce04-baf2-4118-8fab-7518c82b5dd6\") " Jul 13 17:51:54.463769 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:51:54.463646 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e13ce04-baf2-4118-8fab-7518c82b5dd6-tmp-dir\") pod \"5e13ce04-baf2-4118-8fab-7518c82b5dd6\" (UID: \"5e13ce04-baf2-4118-8fab-7518c82b5dd6\") " Jul 13 17:51:54.463769 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:51:54.463711 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e13ce04-baf2-4118-8fab-7518c82b5dd6-home\") pod \"5e13ce04-baf2-4118-8fab-7518c82b5dd6\" (UID: \"5e13ce04-baf2-4118-8fab-7518c82b5dd6\") " Jul 13 17:51:54.463769 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:51:54.463748 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e13ce04-baf2-4118-8fab-7518c82b5dd6-tls-certs\") pod \"5e13ce04-baf2-4118-8fab-7518c82b5dd6\" (UID: \"5e13ce04-baf2-4118-8fab-7518c82b5dd6\") " Jul 13 17:51:54.463931 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:51:54.463792 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e13ce04-baf2-4118-8fab-7518c82b5dd6-model-cache\") pod \"5e13ce04-baf2-4118-8fab-7518c82b5dd6\" (UID: \"5e13ce04-baf2-4118-8fab-7518c82b5dd6\") " Jul 13 17:51:54.463931 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:51:54.463829 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e13ce04-baf2-4118-8fab-7518c82b5dd6-dshm\") pod \"5e13ce04-baf2-4118-8fab-7518c82b5dd6\" (UID: \"5e13ce04-baf2-4118-8fab-7518c82b5dd6\") " Jul 13 17:51:54.464052 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:51:54.463938 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e13ce04-baf2-4118-8fab-7518c82b5dd6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5e13ce04-baf2-4118-8fab-7518c82b5dd6" (UID: "5e13ce04-baf2-4118-8fab-7518c82b5dd6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:51:54.464052 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:51:54.464004 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e13ce04-baf2-4118-8fab-7518c82b5dd6-home" (OuterVolumeSpecName: "home") pod "5e13ce04-baf2-4118-8fab-7518c82b5dd6" (UID: "5e13ce04-baf2-4118-8fab-7518c82b5dd6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:51:54.464173 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:51:54.464105 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e13ce04-baf2-4118-8fab-7518c82b5dd6-model-cache" (OuterVolumeSpecName: "model-cache") pod "5e13ce04-baf2-4118-8fab-7518c82b5dd6" (UID: "5e13ce04-baf2-4118-8fab-7518c82b5dd6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:51:54.464173 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:51:54.464124 2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e13ce04-baf2-4118-8fab-7518c82b5dd6-tmp-dir\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:51:54.464173 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:51:54.464142 2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e13ce04-baf2-4118-8fab-7518c82b5dd6-home\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:51:54.465651 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:51:54.465626 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e13ce04-baf2-4118-8fab-7518c82b5dd6-kube-api-access-dcjm6" (OuterVolumeSpecName: "kube-api-access-dcjm6") pod "5e13ce04-baf2-4118-8fab-7518c82b5dd6" (UID: "5e13ce04-baf2-4118-8fab-7518c82b5dd6"). InnerVolumeSpecName "kube-api-access-dcjm6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:51:54.465855 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:51:54.465830 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e13ce04-baf2-4118-8fab-7518c82b5dd6-dshm" (OuterVolumeSpecName: "dshm") pod "5e13ce04-baf2-4118-8fab-7518c82b5dd6" (UID: "5e13ce04-baf2-4118-8fab-7518c82b5dd6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:51:54.465944 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:51:54.465923 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e13ce04-baf2-4118-8fab-7518c82b5dd6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5e13ce04-baf2-4118-8fab-7518c82b5dd6" (UID: "5e13ce04-baf2-4118-8fab-7518c82b5dd6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 17:51:54.564882 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:51:54.564820 2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e13ce04-baf2-4118-8fab-7518c82b5dd6-tls-certs\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:51:54.564882 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:51:54.564840 2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e13ce04-baf2-4118-8fab-7518c82b5dd6-model-cache\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:51:54.564882 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:51:54.564848 2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e13ce04-baf2-4118-8fab-7518c82b5dd6-dshm\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:51:54.564882 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:51:54.564858 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dcjm6\" (UniqueName: \"kubernetes.io/projected/5e13ce04-baf2-4118-8fab-7518c82b5dd6-kube-api-access-dcjm6\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:51:54.632345 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:51:54.632307 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-714452/rate-test-model-714452-kserve-54b459cbcf-wx8kj"] Jul 13 17:51:54.636680 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:51:54.636659 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-714452/rate-test-model-714452-kserve-54b459cbcf-wx8kj"] Jul 13 17:51:56.137038 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:51:56.137001 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e13ce04-baf2-4118-8fab-7518c82b5dd6" path="/var/lib/kubelet/pods/5e13ce04-baf2-4118-8fab-7518c82b5dd6/volumes" Jul 13 17:51:59.157574 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:51:59.157526 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-714452-dfntf"] Jul 13 17:51:59.158058 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:51:59.158039 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0749e5ff-a060-4c1a-ac8e-97d247fd0770" containerName="main" Jul 13 17:51:59.158106 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:51:59.158061 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="0749e5ff-a060-4c1a-ac8e-97d247fd0770" containerName="main" Jul 13 17:51:59.158106 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:51:59.158088 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e13ce04-baf2-4118-8fab-7518c82b5dd6" containerName="main" Jul 13 17:51:59.158106 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:51:59.158097 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e13ce04-baf2-4118-8fab-7518c82b5dd6" containerName="main" Jul 13 17:51:59.158208 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:51:59.158190 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e13ce04-baf2-4118-8fab-7518c82b5dd6" containerName="main" Jul 13 17:51:59.158208 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:51:59.158205 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="0749e5ff-a060-4c1a-ac8e-97d247fd0770" containerName="main" Jul 13 17:51:59.163427 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:51:59.163409 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-714452-dfntf" Jul 13 17:51:59.165931 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:51:59.165860 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k25z5\"" Jul 13 17:51:59.166137 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:51:59.166121 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 13 17:51:59.168419 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:51:59.168393 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-714452-dfntf"] Jul 13 17:51:59.199452 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:51:59.199423 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e36148c-fcd9-4213-a7ff-5000ec59bdf3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-714452-dfntf\" (UID: \"5e36148c-fcd9-4213-a7ff-5000ec59bdf3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-714452-dfntf" Jul 13 17:51:59.300327 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:51:59.300300 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e36148c-fcd9-4213-a7ff-5000ec59bdf3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-714452-dfntf\" (UID: \"5e36148c-fcd9-4213-a7ff-5000ec59bdf3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-714452-dfntf" Jul 13 17:51:59.300940 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:51:59.300916 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e36148c-fcd9-4213-a7ff-5000ec59bdf3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-714452-dfntf\" (UID: \"5e36148c-fcd9-4213-a7ff-5000ec59bdf3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-714452-dfntf" Jul 13 17:51:59.475550 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:51:59.475489 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-714452-dfntf" Jul 13 17:51:59.614934 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:51:59.614904 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-714452-dfntf"] Jul 13 17:51:59.618729 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:51:59.618701 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e36148c_fcd9_4213_a7ff_5000ec59bdf3.slice/crio-f6169179cfbd10d02d508b9e3a9f90907db4066f20446255ccf9e3ccf1c9d16d WatchSource:0}: Error finding container f6169179cfbd10d02d508b9e3a9f90907db4066f20446255ccf9e3ccf1c9d16d: Status 404 returned error can't find the container with id f6169179cfbd10d02d508b9e3a9f90907db4066f20446255ccf9e3ccf1c9d16d Jul 13 17:52:00.336220 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:52:00.336183 2569 generic.go:356] "Generic (PLEG): container finished" podID="5e36148c-fcd9-4213-a7ff-5000ec59bdf3" containerID="6c40f37a44d84e8e8844ee5d823256bdb5bc7878306ad47c5c54001efccfb528" exitCode=0 Jul 13 17:52:00.336593 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:52:00.336237 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-714452-dfntf" event={"ID":"5e36148c-fcd9-4213-a7ff-5000ec59bdf3","Type":"ContainerDied","Data":"6c40f37a44d84e8e8844ee5d823256bdb5bc7878306ad47c5c54001efccfb528"} Jul 13 17:52:00.336593 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:52:00.336279 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-714452-dfntf" event={"ID":"5e36148c-fcd9-4213-a7ff-5000ec59bdf3","Type":"ContainerStarted","Data":"f6169179cfbd10d02d508b9e3a9f90907db4066f20446255ccf9e3ccf1c9d16d"} Jul 13 17:52:01.469160 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:52:01.469132 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-714452-dfntf" Jul 13 17:52:01.515023 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:52:01.514994 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e36148c-fcd9-4213-a7ff-5000ec59bdf3-maas-api-service-ca\") pod \"5e36148c-fcd9-4213-a7ff-5000ec59bdf3\" (UID: \"5e36148c-fcd9-4213-a7ff-5000ec59bdf3\") " Jul 13 17:52:01.515327 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:52:01.515303 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e36148c-fcd9-4213-a7ff-5000ec59bdf3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5e36148c-fcd9-4213-a7ff-5000ec59bdf3" (UID: "5e36148c-fcd9-4213-a7ff-5000ec59bdf3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:52:01.616174 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:52:01.616107 2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e36148c-fcd9-4213-a7ff-5000ec59bdf3-maas-api-service-ca\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:52:02.346708 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:52:02.346680 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-714452-dfntf" Jul 13 17:52:02.346867 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:52:02.346707 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-714452-dfntf" event={"ID":"5e36148c-fcd9-4213-a7ff-5000ec59bdf3","Type":"ContainerDied","Data":"f6169179cfbd10d02d508b9e3a9f90907db4066f20446255ccf9e3ccf1c9d16d"} Jul 13 17:52:02.346867 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:52:02.346751 2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f6169179cfbd10d02d508b9e3a9f90907db4066f20446255ccf9e3ccf1c9d16d" Jul 13 17:52:25.403597 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:52:25.403531 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-31a784-xvzn7"] Jul 13 17:52:25.404184 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:52:25.404099 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e36148c-fcd9-4213-a7ff-5000ec59bdf3" containerName="revoke-keys" Jul 13 17:52:25.404184 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:52:25.404121 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e36148c-fcd9-4213-a7ff-5000ec59bdf3" containerName="revoke-keys" Jul 13 17:52:25.404314 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:52:25.404208 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e36148c-fcd9-4213-a7ff-5000ec59bdf3" containerName="revoke-keys" Jul 13 17:52:25.407430 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:52:25.407409 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-31a784-xvzn7" Jul 13 17:52:25.410893 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:52:25.410876 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 13 17:52:25.410991 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:52:25.410880 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k25z5\"" Jul 13 17:52:25.416117 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:52:25.416098 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-31a784-xvzn7"] Jul 13 17:52:25.506707 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:52:25.506670 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a27ce954-9086-4da2-8218-8acbdb02d6c9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-31a784-xvzn7\" (UID: \"a27ce954-9086-4da2-8218-8acbdb02d6c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-31a784-xvzn7" Jul 13 17:52:25.607884 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:52:25.607849 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a27ce954-9086-4da2-8218-8acbdb02d6c9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-31a784-xvzn7\" (UID: \"a27ce954-9086-4da2-8218-8acbdb02d6c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-31a784-xvzn7" Jul 13 17:52:25.608457 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:52:25.608437 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a27ce954-9086-4da2-8218-8acbdb02d6c9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-31a784-xvzn7\" (UID: \"a27ce954-9086-4da2-8218-8acbdb02d6c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-31a784-xvzn7" Jul 13 17:52:25.719657 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:52:25.719589 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-31a784-xvzn7" Jul 13 17:52:25.854813 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:52:25.854785 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-31a784-xvzn7"] Jul 13 17:52:25.856596 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:52:25.856531 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda27ce954_9086_4da2_8218_8acbdb02d6c9.slice/crio-4b024c6db65e910c7f223df627c352aa99167216f5c62c4d661ab3eb1713d187 WatchSource:0}: Error finding container 4b024c6db65e910c7f223df627c352aa99167216f5c62c4d661ab3eb1713d187: Status 404 returned error can't find the container with id 4b024c6db65e910c7f223df627c352aa99167216f5c62c4d661ab3eb1713d187 Jul 13 17:52:26.471072 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:52:26.470968 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-31a784-xvzn7" event={"ID":"a27ce954-9086-4da2-8218-8acbdb02d6c9","Type":"ContainerStarted","Data":"5dbb15b2baefb80ec26c6dfd48dd7dcb198cd8b8f5877564f96eed866f0d98f2"} Jul 13 17:52:26.471072 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:52:26.471029 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-31a784-xvzn7" event={"ID":"a27ce954-9086-4da2-8218-8acbdb02d6c9","Type":"ContainerStarted","Data":"4b024c6db65e910c7f223df627c352aa99167216f5c62c4d661ab3eb1713d187"} Jul 13 17:52:26.492444 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:52:26.492385 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-31a784-xvzn7" podStartSLOduration=1.492370798 podStartE2EDuration="1.492370798s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:52:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:52:26.49069566 +0000 UTC m=+2842.888354040" watchObservedRunningTime="2026-07-13 17:52:26.492370798 +0000 UTC m=+2842.890029177" Jul 13 17:52:27.477368 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:52:27.477335 2569 generic.go:356] "Generic (PLEG): container finished" podID="a27ce954-9086-4da2-8218-8acbdb02d6c9" containerID="5dbb15b2baefb80ec26c6dfd48dd7dcb198cd8b8f5877564f96eed866f0d98f2" exitCode=0 Jul 13 17:52:27.477766 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:52:27.477416 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-31a784-xvzn7" event={"ID":"a27ce954-9086-4da2-8218-8acbdb02d6c9","Type":"ContainerDied","Data":"5dbb15b2baefb80ec26c6dfd48dd7dcb198cd8b8f5877564f96eed866f0d98f2"} Jul 13 17:52:28.616936 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:52:28.616902 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-31a784-xvzn7" Jul 13 17:52:28.732507 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:52:28.732479 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a27ce954-9086-4da2-8218-8acbdb02d6c9-maas-api-service-ca\") pod \"a27ce954-9086-4da2-8218-8acbdb02d6c9\" (UID: \"a27ce954-9086-4da2-8218-8acbdb02d6c9\") " Jul 13 17:52:28.732862 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:52:28.732814 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a27ce954-9086-4da2-8218-8acbdb02d6c9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a27ce954-9086-4da2-8218-8acbdb02d6c9" (UID: "a27ce954-9086-4da2-8218-8acbdb02d6c9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:52:28.833459 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:52:28.833435 2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a27ce954-9086-4da2-8218-8acbdb02d6c9-maas-api-service-ca\") on node \"ip-10-0-136-220.ec2.internal\" DevicePath \"\"" Jul 13 17:52:29.488796 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:52:29.488757 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-31a784-xvzn7" event={"ID":"a27ce954-9086-4da2-8218-8acbdb02d6c9","Type":"ContainerDied","Data":"4b024c6db65e910c7f223df627c352aa99167216f5c62c4d661ab3eb1713d187"} Jul 13 17:52:29.488796 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:52:29.488800 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-31a784-xvzn7" Jul 13 17:52:29.489017 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:52:29.488818 2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4b024c6db65e910c7f223df627c352aa99167216f5c62c4d661ab3eb1713d187" Jul 13 17:52:57.351379 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:52:57.351300 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-db977844c-wvpjx_2dd1f384-9047-4f6f-9865-4138986a2532/manager/0.log" Jul 13 17:52:57.849489 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:52:57.849465 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dfb6c5f74-q452h_d5df4bee-43d7-4d5d-bc22-70b20478df18/copy-manifests/0.log" Jul 13 17:52:57.857448 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:52:57.857420 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dfb6c5f74-q452h_d5df4bee-43d7-4d5d-bc22-70b20478df18/manager/0.log" Jul 13 17:52:58.604068 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:52:58.604038 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29732745-2x6fr_e1dce35e-1d10-48d9-ab31-ca7014c4d960/cleanup/0.log" Jul 13 17:52:58.943751 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:52:58.943666 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-a-edd1bf-xqwvb_9b07c283-6c14-48f6-97ba-a5ce2f12529d/revoke-keys/0.log" Jul 13 17:52:59.061394 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:52:59.061369 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-b-e386c8-s2w6c_589b44c1-8180-4474-89a6-e44a6e8fd53f/revoke-keys/0.log" Jul 13 17:52:59.417714 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:52:59.417685 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-b-587790-xrnm7_77eb804a-4260-4fb6-b6a9-1d7d305cef35/revoke-keys/0.log" Jul 13 17:52:59.780424 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:52:59.780354 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-a-714452-dfntf_5e36148c-fcd9-4213-a7ff-5000ec59bdf3/revoke-keys/0.log" Jul 13 17:52:59.895945 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:52:59.895917 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-b-31a784-xvzn7_a27ce954-9086-4da2-8218-8acbdb02d6c9/revoke-keys/0.log" Jul 13 17:53:01.555492 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:01.555455 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-bg5np_d2a9c410-87d9-4acc-ab9d-3d4ffa017ed5/manager/0.log" Jul 13 17:53:01.674762 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:01.674736 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-vhrj6_5784681e-3fbe-4ffe-b796-9cd3589847d2/manager/0.log" Jul 13 17:53:01.918896 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:01.918816 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-d9njc_0b9da11a-0dfd-42e1-891e-2891173db7fa/registry-server/0.log" Jul 13 17:53:02.288369 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:02.288341 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-jxm7m_5541c29b-dcd6-4513-9dde-24f95ed624db/manager/0.log" Jul 13 17:53:03.375551 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:03.375522 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-t2l47_e32dc0d2-f8c6-4ca8-a90f-ae59ff574fa1/payload-processing/0.log" Jul 13 17:53:03.490666 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:03.490638 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-969d568db-zxb2r_e443fc05-8996-474b-85fa-f3a28463785a/router/0.log" Jul 13 17:53:03.837395 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:03.837365 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-ntcvc_895459f5-1d2d-411c-8aa1-357bc8116f85/main/0.log" Jul 13 17:53:04.329652 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:04.329630 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-49fj9_db3344e0-315b-48f3-96f5-543fdfdf024b/main/0.log" Jul 13 17:53:04.441248 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:04.441223 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-vnw7h_bd482960-43cd-4e01-89a3-12e37fdf44c9/main/0.log" Jul 13 17:53:11.573866 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:11.573837 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-nfjpb_2263bf98-5f88-41d9-b755-9f874a3d5916/global-pull-secret-syncer/0.log" Jul 13 17:53:11.705036 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:11.705008 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-phcxr_b3ec2a99-eadb-4865-ae2a-dd77aab461de/konnectivity-agent/0.log" Jul 13 17:53:11.755830 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:11.755800 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-220.ec2.internal_45606745bd8d4c83782613a3e901378b/haproxy/0.log" Jul 13 17:53:16.860103 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:16.860077 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-bg5np_d2a9c410-87d9-4acc-ab9d-3d4ffa017ed5/manager/0.log" Jul 13 17:53:16.885366 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:16.885339 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-vhrj6_5784681e-3fbe-4ffe-b796-9cd3589847d2/manager/0.log" Jul 13 17:53:16.957372 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:16.957342 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-d9njc_0b9da11a-0dfd-42e1-891e-2891173db7fa/registry-server/0.log" Jul 13 17:53:17.213127 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:17.213047 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-jxm7m_5541c29b-dcd6-4513-9dde-24f95ed624db/manager/0.log" Jul 13 17:53:19.015882 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:19.015859 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-ttd4d_7549ca91-bd0b-4b2f-b76c-0a7784f2fd4c/cluster-monitoring-operator/0.log" Jul 13 17:53:19.346157 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:19.346134 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qrt8r_41a0e37d-a408-4cf1-946d-d573d2d5cfa4/node-exporter/0.log" Jul 13 17:53:19.369433 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:19.369410 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qrt8r_41a0e37d-a408-4cf1-946d-d573d2d5cfa4/kube-rbac-proxy/0.log" Jul 13 17:53:19.392838 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:19.392818 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qrt8r_41a0e37d-a408-4cf1-946d-d573d2d5cfa4/init-textfile/0.log" Jul 13 17:53:21.219911 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:21.219879 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-5wvjq_afb76ab4-2a66-45a7-8c98-7c3ca6e026b9/networking-console-plugin/0.log" Jul 13 17:53:21.791083 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:21.791054 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-rq8n6/perf-node-gather-daemonset-lgdpp"] Jul 13 17:53:21.791433 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:21.791421 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a27ce954-9086-4da2-8218-8acbdb02d6c9" containerName="revoke-keys" Jul 13 17:53:21.791478 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:21.791435 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="a27ce954-9086-4da2-8218-8acbdb02d6c9" containerName="revoke-keys" Jul 13 17:53:21.791512 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:21.791496 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="a27ce954-9086-4da2-8218-8acbdb02d6c9" containerName="revoke-keys" Jul 13 17:53:21.794590 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:21.794553 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rq8n6/perf-node-gather-daemonset-lgdpp" Jul 13 17:53:21.796497 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:21.796475 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rq8n6\"/\"kube-root-ca.crt\"" Jul 13 17:53:21.796810 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:21.796790 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rq8n6\"/\"openshift-service-ca.crt\"" Jul 13 17:53:21.798630 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:21.798613 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-rq8n6\"/\"default-dockercfg-2c2l5\"" Jul 13 17:53:21.803138 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:21.803116 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rq8n6/perf-node-gather-daemonset-lgdpp"] Jul 13 17:53:21.855630 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:21.855592 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-hzcql_c40e6e88-ab3f-4e3c-a0e7-2400bda333eb/console-operator/1.log" Jul 13 17:53:21.859849 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:21.859830 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-hzcql_c40e6e88-ab3f-4e3c-a0e7-2400bda333eb/console-operator/2.log" Jul 13 17:53:21.862546 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:21.862529 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2ef1c37c-5bc9-4e6e-b255-3a4a1a95a3d9-sys\") pod \"perf-node-gather-daemonset-lgdpp\" (UID: \"2ef1c37c-5bc9-4e6e-b255-3a4a1a95a3d9\") " pod="openshift-must-gather-rq8n6/perf-node-gather-daemonset-lgdpp" Jul 13 17:53:21.862647 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:21.862555 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vj9wc\" (UniqueName: \"kubernetes.io/projected/2ef1c37c-5bc9-4e6e-b255-3a4a1a95a3d9-kube-api-access-vj9wc\") pod \"perf-node-gather-daemonset-lgdpp\" (UID: \"2ef1c37c-5bc9-4e6e-b255-3a4a1a95a3d9\") " pod="openshift-must-gather-rq8n6/perf-node-gather-daemonset-lgdpp" Jul 13 17:53:21.862647 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:21.862587 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2ef1c37c-5bc9-4e6e-b255-3a4a1a95a3d9-podres\") pod \"perf-node-gather-daemonset-lgdpp\" (UID: \"2ef1c37c-5bc9-4e6e-b255-3a4a1a95a3d9\") " pod="openshift-must-gather-rq8n6/perf-node-gather-daemonset-lgdpp" Jul 13 17:53:21.862767 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:21.862646 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2ef1c37c-5bc9-4e6e-b255-3a4a1a95a3d9-proc\") pod \"perf-node-gather-daemonset-lgdpp\" (UID: \"2ef1c37c-5bc9-4e6e-b255-3a4a1a95a3d9\") " pod="openshift-must-gather-rq8n6/perf-node-gather-daemonset-lgdpp" Jul 13 17:53:21.862767 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:21.862687 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2ef1c37c-5bc9-4e6e-b255-3a4a1a95a3d9-lib-modules\") pod \"perf-node-gather-daemonset-lgdpp\" (UID: \"2ef1c37c-5bc9-4e6e-b255-3a4a1a95a3d9\") " pod="openshift-must-gather-rq8n6/perf-node-gather-daemonset-lgdpp" Jul 13 17:53:21.963044 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:21.963019 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2ef1c37c-5bc9-4e6e-b255-3a4a1a95a3d9-sys\") pod \"perf-node-gather-daemonset-lgdpp\" (UID: \"2ef1c37c-5bc9-4e6e-b255-3a4a1a95a3d9\") " pod="openshift-must-gather-rq8n6/perf-node-gather-daemonset-lgdpp" Jul 13 17:53:21.963148 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:21.963047 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vj9wc\" (UniqueName: \"kubernetes.io/projected/2ef1c37c-5bc9-4e6e-b255-3a4a1a95a3d9-kube-api-access-vj9wc\") pod \"perf-node-gather-daemonset-lgdpp\" (UID: \"2ef1c37c-5bc9-4e6e-b255-3a4a1a95a3d9\") " pod="openshift-must-gather-rq8n6/perf-node-gather-daemonset-lgdpp" Jul 13 17:53:21.963148 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:21.963066 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2ef1c37c-5bc9-4e6e-b255-3a4a1a95a3d9-podres\") pod \"perf-node-gather-daemonset-lgdpp\" (UID: \"2ef1c37c-5bc9-4e6e-b255-3a4a1a95a3d9\") " pod="openshift-must-gather-rq8n6/perf-node-gather-daemonset-lgdpp" Jul 13 17:53:21.963148 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:21.963084 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2ef1c37c-5bc9-4e6e-b255-3a4a1a95a3d9-proc\") pod \"perf-node-gather-daemonset-lgdpp\" (UID: \"2ef1c37c-5bc9-4e6e-b255-3a4a1a95a3d9\") " pod="openshift-must-gather-rq8n6/perf-node-gather-daemonset-lgdpp" Jul 13 17:53:21.963148 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:21.963103 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2ef1c37c-5bc9-4e6e-b255-3a4a1a95a3d9-lib-modules\") pod \"perf-node-gather-daemonset-lgdpp\" (UID: \"2ef1c37c-5bc9-4e6e-b255-3a4a1a95a3d9\") " pod="openshift-must-gather-rq8n6/perf-node-gather-daemonset-lgdpp" Jul 13 17:53:21.963301 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:21.963144 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2ef1c37c-5bc9-4e6e-b255-3a4a1a95a3d9-sys\") pod \"perf-node-gather-daemonset-lgdpp\" (UID: \"2ef1c37c-5bc9-4e6e-b255-3a4a1a95a3d9\") " pod="openshift-must-gather-rq8n6/perf-node-gather-daemonset-lgdpp" Jul 13 17:53:21.963301 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:21.963162 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2ef1c37c-5bc9-4e6e-b255-3a4a1a95a3d9-podres\") pod \"perf-node-gather-daemonset-lgdpp\" (UID: \"2ef1c37c-5bc9-4e6e-b255-3a4a1a95a3d9\") " pod="openshift-must-gather-rq8n6/perf-node-gather-daemonset-lgdpp" Jul 13 17:53:21.963301 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:21.963168 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2ef1c37c-5bc9-4e6e-b255-3a4a1a95a3d9-proc\") pod \"perf-node-gather-daemonset-lgdpp\" (UID: \"2ef1c37c-5bc9-4e6e-b255-3a4a1a95a3d9\") " pod="openshift-must-gather-rq8n6/perf-node-gather-daemonset-lgdpp" Jul 13 17:53:21.963301 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:21.963254 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2ef1c37c-5bc9-4e6e-b255-3a4a1a95a3d9-lib-modules\") pod \"perf-node-gather-daemonset-lgdpp\" (UID: \"2ef1c37c-5bc9-4e6e-b255-3a4a1a95a3d9\") " pod="openshift-must-gather-rq8n6/perf-node-gather-daemonset-lgdpp" Jul 13 17:53:21.969828 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:21.969808 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vj9wc\" (UniqueName: \"kubernetes.io/projected/2ef1c37c-5bc9-4e6e-b255-3a4a1a95a3d9-kube-api-access-vj9wc\") pod \"perf-node-gather-daemonset-lgdpp\" (UID: \"2ef1c37c-5bc9-4e6e-b255-3a4a1a95a3d9\") " pod="openshift-must-gather-rq8n6/perf-node-gather-daemonset-lgdpp" Jul 13 17:53:22.105683 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:22.105629 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rq8n6/perf-node-gather-daemonset-lgdpp" Jul 13 17:53:22.237464 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:22.237433 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rq8n6/perf-node-gather-daemonset-lgdpp"] Jul 13 17:53:22.240622 ip-10-0-136-220 kubenswrapper[2569]: W0713 17:53:22.240592 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod2ef1c37c_5bc9_4e6e_b255_3a4a1a95a3d9.slice/crio-07bb8bab8950479c9614716171fac91d8b901a2cbb92bb13490224bd209c0b50 WatchSource:0}: Error finding container 07bb8bab8950479c9614716171fac91d8b901a2cbb92bb13490224bd209c0b50: Status 404 returned error can't find the container with id 07bb8bab8950479c9614716171fac91d8b901a2cbb92bb13490224bd209c0b50 Jul 13 17:53:22.759285 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:22.759247 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rq8n6/perf-node-gather-daemonset-lgdpp" event={"ID":"2ef1c37c-5bc9-4e6e-b255-3a4a1a95a3d9","Type":"ContainerStarted","Data":"6993c751fb073fe5cb0295ea12911599117f5be6afd0f85503bc563f95192c14"} Jul 13 17:53:22.759449 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:22.759295 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rq8n6/perf-node-gather-daemonset-lgdpp" event={"ID":"2ef1c37c-5bc9-4e6e-b255-3a4a1a95a3d9","Type":"ContainerStarted","Data":"07bb8bab8950479c9614716171fac91d8b901a2cbb92bb13490224bd209c0b50"} Jul 13 17:53:22.759449 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:22.759327 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-rq8n6/perf-node-gather-daemonset-lgdpp" Jul 13 17:53:22.775206 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:22.775161 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-rq8n6/perf-node-gather-daemonset-lgdpp" podStartSLOduration=1.775146162 podStartE2EDuration="1.775146162s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 17:53:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:53:22.773725677 +0000 UTC m=+2899.171384055" watchObservedRunningTime="2026-07-13 17:53:22.775146162 +0000 UTC m=+2899.172804541" Jul 13 17:53:23.073273 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:23.073247 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-rhgb9_2bfc2ce7-6ebd-4197-86fa-19d9e2b385c8/volume-data-source-validator/0.log" Jul 13 17:53:24.135691 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:24.135662 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9xwzd_57b96b57-d3e5-44f3-bcca-2a077820b845/dns/0.log" Jul 13 17:53:24.180818 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:24.180791 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9xwzd_57b96b57-d3e5-44f3-bcca-2a077820b845/kube-rbac-proxy/0.log" Jul 13 17:53:24.231343 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:24.231318 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-qn7kc_a7a4b481-50e3-4bb3-bcb5-6c5942814726/dns-node-resolver/0.log" Jul 13 17:53:24.771249 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:24.771218 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-psn5f_99b9c479-5323-44a6-bbfa-2aa301efb6af/node-ca/0.log" Jul 13 17:53:26.039318 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:26.039288 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-t2l47_e32dc0d2-f8c6-4ca8-a90f-ae59ff574fa1/payload-processing/0.log" Jul 13 17:53:26.060512 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:26.060485 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-969d568db-zxb2r_e443fc05-8996-474b-85fa-f3a28463785a/router/0.log" Jul 13 17:53:26.727806 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:26.727782 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5s55t_2f125df3-6cb8-4ec4-bd2f-24428ce6127e/serve-healthcheck-canary/0.log" Jul 13 17:53:27.223226 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:27.223199 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-j2xl7_7ffdcb20-b7c8-468e-9c1f-c68e9e588a93/insights-operator/0.log" Jul 13 17:53:27.224003 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:27.223988 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-j2xl7_7ffdcb20-b7c8-468e-9c1f-c68e9e588a93/insights-operator/1.log" Jul 13 17:53:27.314364 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:27.314341 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cfcv4_965ad197-c001-48c3-9ffd-58b4d7415d1e/kube-rbac-proxy/0.log" Jul 13 17:53:27.334741 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:27.334717 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cfcv4_965ad197-c001-48c3-9ffd-58b4d7415d1e/exporter/0.log" Jul 13 17:53:27.355317 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:27.355297 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cfcv4_965ad197-c001-48c3-9ffd-58b4d7415d1e/extractor/0.log" Jul 13 17:53:28.774356 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:28.774329 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-rq8n6/perf-node-gather-daemonset-lgdpp" Jul 13 17:53:29.795370 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:29.795323 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-db977844c-wvpjx_2dd1f384-9047-4f6f-9865-4138986a2532/manager/0.log" Jul 13 17:53:30.047535 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:30.047454 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dfb6c5f74-q452h_d5df4bee-43d7-4d5d-bc22-70b20478df18/manager/0.log" Jul 13 17:53:30.068668 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:30.068643 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dfb6c5f74-q452h_d5df4bee-43d7-4d5d-bc22-70b20478df18/copy-manifests/0.log" Jul 13 17:53:31.591921 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:31.591895 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-f6649c567-rhl5t_4aa024cf-46c7-429a-b180-ef267e7b69b5/manager/0.log" Jul 13 17:53:32.556367 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:32.556335 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-t2l47_e32dc0d2-f8c6-4ca8-a90f-ae59ff574fa1/payload-processing/0.log" Jul 13 17:53:32.573829 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:32.573804 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-969d568db-zxb2r_e443fc05-8996-474b-85fa-f3a28463785a/router/0.log" Jul 13 17:53:34.083046 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:34.083011 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-t2l47_e32dc0d2-f8c6-4ca8-a90f-ae59ff574fa1/payload-processing/0.log" Jul 13 17:53:34.096144 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:34.096119 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-969d568db-zxb2r_e443fc05-8996-474b-85fa-f3a28463785a/router/0.log" Jul 13 17:53:36.637649 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:36.637616 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-9pnpw_4e92fb26-ea9b-4a99-bc67-5e52b023aa20/migrator/0.log" Jul 13 17:53:36.658065 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:36.658039 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-9pnpw_4e92fb26-ea9b-4a99-bc67-5e52b023aa20/graceful-termination/0.log" Jul 13 17:53:36.997899 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:36.997816 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-7s9dh_bd4ce4fa-6339-4a0c-8260-c6a9538543c2/kube-storage-version-migrator-operator/1.log" Jul 13 17:53:36.998679 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:36.998657 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-7s9dh_bd4ce4fa-6339-4a0c-8260-c6a9538543c2/kube-storage-version-migrator-operator/0.log" Jul 13 17:53:38.381537 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:38.381507 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xpws4_921a5cfd-83c7-4db7-be13-53e44c5df655/kube-multus-additional-cni-plugins/0.log" Jul 13 17:53:38.402971 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:38.402944 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xpws4_921a5cfd-83c7-4db7-be13-53e44c5df655/egress-router-binary-copy/0.log" Jul 13 17:53:38.424730 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:38.424702 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xpws4_921a5cfd-83c7-4db7-be13-53e44c5df655/cni-plugins/0.log" Jul 13 17:53:38.445757 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:38.445733 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xpws4_921a5cfd-83c7-4db7-be13-53e44c5df655/bond-cni-plugin/0.log" Jul 13 17:53:38.466407 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:38.466384 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xpws4_921a5cfd-83c7-4db7-be13-53e44c5df655/routeoverride-cni/0.log" Jul 13 17:53:38.486602 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:38.486576 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xpws4_921a5cfd-83c7-4db7-be13-53e44c5df655/whereabouts-cni-bincopy/0.log" Jul 13 17:53:38.507494 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:38.507479 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xpws4_921a5cfd-83c7-4db7-be13-53e44c5df655/whereabouts-cni/0.log" Jul 13 17:53:38.539442 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:38.539416 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-cwgjn_252eeac8-ef53-439f-8d2f-8ef03a102104/kube-multus/0.log" Jul 13 17:53:38.631464 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:38.631446 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-t6r98_7cb8e14a-393f-43f2-b75a-7f14756cfb2a/network-metrics-daemon/0.log" Jul 13 17:53:38.650823 ip-10-0-136-220 kubenswrapper[2569]: I0713 17:53:38.650769 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-t6r98_7cb8e14a-393f-43f2-b75a-7f14756cfb2a/kube-rbac-proxy/0.log"