Jul 17 07:09:56.625393 ip-10-0-138-233 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 17 07:09:56.625407 ip-10-0-138-233 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 17 07:09:56.625417 ip-10-0-138-233 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 17 07:09:56.625724 ip-10-0-138-233 systemd[1]: Failed to start Kubernetes Kubelet. Jul 17 07:10:06.635797 ip-10-0-138-233 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 17 07:10:06.635810 ip-10-0-138-233 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot c4fe21ecedee4f41b07f74645e14afb3 -- Jul 17 07:11:34.349694 ip-10-0-138-233 systemd[1]: Starting Kubernetes Kubelet... Jul 17 07:11:34.791597 ip-10-0-138-233 kubenswrapper[2568]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 17 07:11:34.791597 ip-10-0-138-233 kubenswrapper[2568]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 17 07:11:34.791597 ip-10-0-138-233 kubenswrapper[2568]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 17 07:11:34.791597 ip-10-0-138-233 kubenswrapper[2568]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 17 07:11:34.792172 ip-10-0-138-233 kubenswrapper[2568]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 17 07:11:34.793663 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.793581 2568 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 17 07:11:34.799290 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799276 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 17 07:11:34.799328 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799292 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 17 07:11:34.799328 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799297 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 17 07:11:34.799328 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799300 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 17 07:11:34.799328 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799303 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 17 07:11:34.799328 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799306 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 17 07:11:34.799328 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799309 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 17 07:11:34.799328 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799312 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 17 07:11:34.799328 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799314 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 17 07:11:34.799328 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799317 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 17 07:11:34.799328 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799320 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 17 07:11:34.799328 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799322 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 17 07:11:34.799328 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799325 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 17 07:11:34.799328 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799328 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 17 07:11:34.799328 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799330 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 17 07:11:34.799328 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799333 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 17 07:11:34.799700 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799336 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 17 07:11:34.799700 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799339 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 17 07:11:34.799700 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799341 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 17 07:11:34.799700 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799345 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 17 07:11:34.799700 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799348 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 17 07:11:34.799700 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799350 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 17 07:11:34.799700 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799353 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 17 07:11:34.799700 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799355 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 17 07:11:34.799700 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799358 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 17 07:11:34.799700 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799360 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 17 07:11:34.799700 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799363 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 17 07:11:34.799700 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799366 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 17 07:11:34.799700 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799368 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 17 07:11:34.799700 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799371 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 17 07:11:34.799700 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799374 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 17 07:11:34.799700 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799376 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 17 07:11:34.799700 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799379 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 17 07:11:34.799700 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799381 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 17 07:11:34.799700 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799385 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 17 07:11:34.800169 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799402 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 17 07:11:34.800169 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799406 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 17 07:11:34.800169 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799409 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 17 07:11:34.800169 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799413 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 17 07:11:34.800169 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799415 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 17 07:11:34.800169 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799418 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 17 07:11:34.800169 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799422 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 17 07:11:34.800169 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799424 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 17 07:11:34.800169 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799427 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 17 07:11:34.800169 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799430 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 17 07:11:34.800169 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799433 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 17 07:11:34.800169 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799437 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 17 07:11:34.800169 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799439 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 17 07:11:34.800169 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799442 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 17 07:11:34.800169 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799445 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 17 07:11:34.800169 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799448 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 17 07:11:34.800169 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799451 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 17 07:11:34.800169 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799455 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 17 07:11:34.800169 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799458 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 17 07:11:34.800638 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799461 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 17 07:11:34.800638 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799464 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 17 07:11:34.800638 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799466 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 17 07:11:34.800638 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799469 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 17 07:11:34.800638 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799472 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 17 07:11:34.800638 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799474 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 17 07:11:34.800638 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799477 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 17 07:11:34.800638 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799480 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 17 07:11:34.800638 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799482 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 17 07:11:34.800638 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799485 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 17 07:11:34.800638 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799487 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 17 07:11:34.800638 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799490 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 17 07:11:34.800638 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799492 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 17 07:11:34.800638 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799495 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 17 07:11:34.800638 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799497 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 17 07:11:34.800638 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799500 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 17 07:11:34.800638 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799503 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 17 07:11:34.800638 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799507 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 17 07:11:34.800638 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799510 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 17 07:11:34.800638 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799513 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 17 07:11:34.801197 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799515 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 17 07:11:34.801197 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799518 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 17 07:11:34.801197 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799520 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 17 07:11:34.801197 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799523 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 17 07:11:34.801197 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799525 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 17 07:11:34.801197 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799528 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 17 07:11:34.801197 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799530 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 17 07:11:34.801197 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799534 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 17 07:11:34.801197 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799536 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 17 07:11:34.801197 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799539 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 17 07:11:34.801197 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799542 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 17 07:11:34.801197 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799544 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 17 07:11:34.801197 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799547 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 17 07:11:34.801197 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799549 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 17 07:11:34.801197 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799552 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 17 07:11:34.801197 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799555 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 17 07:11:34.801197 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799557 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 17 07:11:34.801197 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799559 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 17 07:11:34.801197 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799562 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 17 07:11:34.801197 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799564 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 17 07:11:34.801716 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799567 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 17 07:11:34.801716 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799569 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 17 07:11:34.801716 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799572 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 17 07:11:34.801716 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799574 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 17 07:11:34.801716 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.799577 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 17 07:11:34.801716 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800294 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 17 07:11:34.801716 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800301 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 17 07:11:34.801716 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800304 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 17 07:11:34.801716 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800306 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 17 07:11:34.801716 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800309 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 17 07:11:34.801716 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800313 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 17 07:11:34.801716 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800315 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 17 07:11:34.801716 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800318 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 17 07:11:34.801716 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800321 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 17 07:11:34.801716 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800323 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 17 07:11:34.801716 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800326 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 17 07:11:34.801716 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800328 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 17 07:11:34.801716 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800331 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 17 07:11:34.801716 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800334 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 17 07:11:34.801716 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800337 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 17 07:11:34.802209 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800339 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 17 07:11:34.802209 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800342 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 17 07:11:34.802209 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800345 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 17 07:11:34.802209 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800347 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 17 07:11:34.802209 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800350 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 17 07:11:34.802209 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800352 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 17 07:11:34.802209 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800355 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 17 07:11:34.802209 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800357 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 17 07:11:34.802209 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800360 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 17 07:11:34.802209 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800363 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 17 07:11:34.802209 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800365 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 17 07:11:34.802209 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800367 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 17 07:11:34.802209 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800370 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 17 07:11:34.802209 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800373 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 17 07:11:34.802209 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800375 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 17 07:11:34.802209 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800377 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 17 07:11:34.802209 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800380 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 17 07:11:34.802209 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800382 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 17 07:11:34.802209 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800385 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 17 07:11:34.802209 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800402 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 17 07:11:34.802707 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800406 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 17 07:11:34.802707 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800409 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 17 07:11:34.802707 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800411 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 17 07:11:34.802707 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800414 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 17 07:11:34.802707 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800417 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 17 07:11:34.802707 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800420 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 17 07:11:34.802707 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800422 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 17 07:11:34.802707 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800425 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 17 07:11:34.802707 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800428 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 17 07:11:34.802707 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800430 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 17 07:11:34.802707 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800433 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 17 07:11:34.802707 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800435 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 17 07:11:34.802707 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800438 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 17 07:11:34.802707 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800440 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 17 07:11:34.802707 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800443 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 17 07:11:34.802707 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800445 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 17 07:11:34.802707 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800448 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 17 07:11:34.802707 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800450 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 17 07:11:34.802707 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800453 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 17 07:11:34.802707 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800456 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 17 07:11:34.803210 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800459 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 17 07:11:34.803210 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800461 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 17 07:11:34.803210 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800464 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 17 07:11:34.803210 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800466 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 17 07:11:34.803210 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800470 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 17 07:11:34.803210 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800472 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 17 07:11:34.803210 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800475 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 17 07:11:34.803210 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800477 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 17 07:11:34.803210 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800480 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 17 07:11:34.803210 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800482 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 17 07:11:34.803210 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800484 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 17 07:11:34.803210 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800488 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 17 07:11:34.803210 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800492 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 17 07:11:34.803210 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800495 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 17 07:11:34.803210 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800498 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 17 07:11:34.803210 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800502 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 17 07:11:34.803210 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800505 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 17 07:11:34.803210 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800508 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 17 07:11:34.803210 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800511 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 17 07:11:34.803210 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800513 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 17 07:11:34.803713 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800516 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 17 07:11:34.803713 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800518 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 17 07:11:34.803713 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800521 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 17 07:11:34.803713 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800523 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 17 07:11:34.803713 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800526 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 17 07:11:34.803713 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800528 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 17 07:11:34.803713 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800531 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 17 07:11:34.803713 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800533 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 17 07:11:34.803713 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800538 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 17 07:11:34.803713 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800540 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 17 07:11:34.803713 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800543 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 17 07:11:34.803713 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800546 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 17 07:11:34.803713 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800549 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 17 07:11:34.803713 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800552 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 17 07:11:34.803713 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800556 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 17 07:11:34.803713 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800559 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 17 07:11:34.803713 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800562 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 17 07:11:34.803713 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800565 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 17 07:11:34.803713 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800568 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 17 07:11:34.804200 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800572 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 17 07:11:34.804200 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800575 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 17 07:11:34.804200 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800578 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 17 07:11:34.804200 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800581 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 17 07:11:34.804200 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800584 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 17 07:11:34.804200 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800945 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 17 07:11:34.804200 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800950 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 17 07:11:34.804200 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800952 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 17 07:11:34.804200 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800956 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 17 07:11:34.804200 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800959 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 17 07:11:34.804200 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800962 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 17 07:11:34.804200 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800965 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 17 07:11:34.804200 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800968 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 17 07:11:34.804200 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800970 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 17 07:11:34.804200 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800973 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 17 07:11:34.804200 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800975 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 17 07:11:34.804200 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800978 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 17 07:11:34.804200 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800980 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 17 07:11:34.804200 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800983 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 17 07:11:34.804660 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800986 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 17 07:11:34.804660 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800988 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 17 07:11:34.804660 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800993 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 17 07:11:34.804660 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800995 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 17 07:11:34.804660 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.800998 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 17 07:11:34.804660 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801000 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 17 07:11:34.804660 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801003 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 17 07:11:34.804660 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801005 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 17 07:11:34.804660 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801008 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 17 07:11:34.804660 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801010 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 17 07:11:34.804660 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801013 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 17 07:11:34.804660 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801015 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 17 07:11:34.804660 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801018 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 17 07:11:34.804660 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801020 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 17 07:11:34.804660 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801023 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 17 07:11:34.804660 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801027 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 17 07:11:34.804660 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801030 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 17 07:11:34.804660 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801033 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 17 07:11:34.804660 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801036 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 17 07:11:34.804660 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801039 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 17 07:11:34.805133 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801042 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 17 07:11:34.805133 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801044 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 17 07:11:34.805133 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801047 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 17 07:11:34.805133 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801050 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 17 07:11:34.805133 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801053 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 17 07:11:34.805133 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801057 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 17 07:11:34.805133 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801059 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 17 07:11:34.805133 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801062 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 17 07:11:34.805133 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801065 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 17 07:11:34.805133 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801067 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 17 07:11:34.805133 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801071 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 17 07:11:34.805133 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801074 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 17 07:11:34.805133 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801077 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 17 07:11:34.805133 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801080 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 17 07:11:34.805133 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801084 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 17 07:11:34.805133 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801087 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 17 07:11:34.805133 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801089 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 17 07:11:34.805133 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801093 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 17 07:11:34.805133 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801096 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 17 07:11:34.805634 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801099 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 17 07:11:34.805634 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801101 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 17 07:11:34.805634 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801104 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 17 07:11:34.805634 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801107 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 17 07:11:34.805634 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801109 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 17 07:11:34.805634 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801111 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 17 07:11:34.805634 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801114 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 17 07:11:34.805634 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801117 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 17 07:11:34.805634 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801119 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 17 07:11:34.805634 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801122 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 17 07:11:34.805634 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801124 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 17 07:11:34.805634 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801127 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 17 07:11:34.805634 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801129 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 17 07:11:34.805634 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801132 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 17 07:11:34.805634 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801134 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 17 07:11:34.805634 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801137 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 17 07:11:34.805634 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801139 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 17 07:11:34.805634 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801142 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 17 07:11:34.805634 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801145 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 17 07:11:34.806092 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801147 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 17 07:11:34.806092 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801150 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 17 07:11:34.806092 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801152 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 17 07:11:34.806092 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801155 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 17 07:11:34.806092 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801157 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 17 07:11:34.806092 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801159 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 17 07:11:34.806092 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801162 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 17 07:11:34.806092 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801164 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 17 07:11:34.806092 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801168 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 17 07:11:34.806092 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801170 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 17 07:11:34.806092 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801173 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 17 07:11:34.806092 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801175 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 17 07:11:34.806092 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801177 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 17 07:11:34.806092 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801180 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 17 07:11:34.806092 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801182 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 17 07:11:34.806092 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801185 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 17 07:11:34.806092 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801187 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 17 07:11:34.806092 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801189 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 17 07:11:34.806092 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801192 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 17 07:11:34.806558 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801194 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 17 07:11:34.806558 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801197 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 17 07:11:34.806558 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801200 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 17 07:11:34.806558 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801204 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 17 07:11:34.806558 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801207 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 17 07:11:34.806558 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801209 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 17 07:11:34.806558 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801212 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 17 07:11:34.806558 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801215 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 17 07:11:34.806558 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801287 2568 flags.go:64] FLAG: --address="0.0.0.0" Jul 17 07:11:34.806558 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801298 2568 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 17 07:11:34.806558 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801304 2568 flags.go:64] FLAG: --anonymous-auth="true" Jul 17 07:11:34.806558 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801308 2568 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 17 07:11:34.806558 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801313 2568 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 17 07:11:34.806558 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801316 2568 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 17 07:11:34.806558 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801320 2568 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 17 07:11:34.806558 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801324 2568 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 17 07:11:34.806558 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801327 2568 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 17 07:11:34.806558 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801330 2568 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 17 07:11:34.806558 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801333 2568 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 17 07:11:34.806558 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801336 2568 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 17 07:11:34.806558 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801339 2568 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 17 07:11:34.806558 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801344 2568 flags.go:64] FLAG: --cgroup-root="" Jul 17 07:11:34.807154 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801347 2568 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 17 07:11:34.807154 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801350 2568 flags.go:64] FLAG: --client-ca-file="" Jul 17 07:11:34.807154 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801352 2568 flags.go:64] FLAG: --cloud-provider="external" Jul 17 07:11:34.807154 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801355 2568 flags.go:64] FLAG: --cluster-dns="[]" Jul 17 07:11:34.807154 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801359 2568 flags.go:64] FLAG: --cluster-domain="" Jul 17 07:11:34.807154 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801362 2568 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 17 07:11:34.807154 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801365 2568 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 17 07:11:34.807154 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801368 2568 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 17 07:11:34.807154 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801371 2568 flags.go:64] FLAG: --container-log-max-files="5" Jul 17 07:11:34.807154 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801375 2568 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 17 07:11:34.807154 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801378 2568 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 17 07:11:34.807154 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801381 2568 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 17 07:11:34.807154 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801384 2568 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 17 07:11:34.807154 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801403 2568 flags.go:64] FLAG: --contention-profiling="false" Jul 17 07:11:34.807154 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801406 2568 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 17 07:11:34.807154 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801410 2568 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 17 07:11:34.807154 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801413 2568 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 17 07:11:34.807154 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801416 2568 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 17 07:11:34.807154 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801420 2568 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 17 07:11:34.807154 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801423 2568 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 17 07:11:34.807154 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801426 2568 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 17 07:11:34.807154 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801429 2568 flags.go:64] FLAG: --enable-load-reader="false" Jul 17 07:11:34.807154 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801432 2568 flags.go:64] FLAG: --enable-server="true" Jul 17 07:11:34.807154 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801435 2568 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 17 07:11:34.807154 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801439 2568 flags.go:64] FLAG: --event-burst="100" Jul 17 07:11:34.807759 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801442 2568 flags.go:64] FLAG: --event-qps="50" Jul 17 07:11:34.807759 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801445 2568 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 17 07:11:34.807759 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801448 2568 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 17 07:11:34.807759 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801451 2568 flags.go:64] FLAG: --eviction-hard="" Jul 17 07:11:34.807759 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801454 2568 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 17 07:11:34.807759 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801457 2568 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 17 07:11:34.807759 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801461 2568 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 17 07:11:34.807759 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801464 2568 flags.go:64] FLAG: --eviction-soft="" Jul 17 07:11:34.807759 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801467 2568 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 17 07:11:34.807759 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801470 2568 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 17 07:11:34.807759 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801473 2568 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 17 07:11:34.807759 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801476 2568 flags.go:64] FLAG: --experimental-mounter-path="" Jul 17 07:11:34.807759 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801479 2568 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 17 07:11:34.807759 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801482 2568 flags.go:64] FLAG: --fail-swap-on="true" Jul 17 07:11:34.807759 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801485 2568 flags.go:64] FLAG: --feature-gates="" Jul 17 07:11:34.807759 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801488 2568 flags.go:64] FLAG: --file-check-frequency="20s" Jul 17 07:11:34.807759 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801491 2568 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 17 07:11:34.807759 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801494 2568 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 17 07:11:34.807759 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801497 2568 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 17 07:11:34.807759 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801500 2568 flags.go:64] FLAG: --healthz-port="10248" Jul 17 07:11:34.807759 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801504 2568 flags.go:64] FLAG: --help="false" Jul 17 07:11:34.807759 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801507 2568 flags.go:64] FLAG: --hostname-override="ip-10-0-138-233.ec2.internal" Jul 17 07:11:34.807759 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801510 2568 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 17 07:11:34.807759 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801513 2568 flags.go:64] FLAG: --http-check-frequency="20s" Jul 17 07:11:34.808368 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801516 2568 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 17 07:11:34.808368 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801520 2568 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 17 07:11:34.808368 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801523 2568 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 17 07:11:34.808368 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801526 2568 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 17 07:11:34.808368 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801529 2568 flags.go:64] FLAG: --image-service-endpoint="" Jul 17 07:11:34.808368 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801532 2568 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 17 07:11:34.808368 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801534 2568 flags.go:64] FLAG: --kube-api-burst="100" Jul 17 07:11:34.808368 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801538 2568 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 17 07:11:34.808368 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801541 2568 flags.go:64] FLAG: --kube-api-qps="50" Jul 17 07:11:34.808368 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801544 2568 flags.go:64] FLAG: --kube-reserved="" Jul 17 07:11:34.808368 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801546 2568 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 17 07:11:34.808368 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801549 2568 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 17 07:11:34.808368 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801552 2568 flags.go:64] FLAG: --kubelet-cgroups="" Jul 17 07:11:34.808368 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801555 2568 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 17 07:11:34.808368 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801559 2568 flags.go:64] FLAG: --lock-file="" Jul 17 07:11:34.808368 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801562 2568 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 17 07:11:34.808368 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801565 2568 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 17 07:11:34.808368 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801568 2568 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 17 07:11:34.808368 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801573 2568 flags.go:64] FLAG: --log-json-split-stream="false" Jul 17 07:11:34.808368 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801576 2568 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 17 07:11:34.808368 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801578 2568 flags.go:64] FLAG: --log-text-split-stream="false" Jul 17 07:11:34.808368 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801581 2568 flags.go:64] FLAG: --logging-format="text" Jul 17 07:11:34.808368 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801584 2568 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 17 07:11:34.808960 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801587 2568 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 17 07:11:34.808960 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801590 2568 flags.go:64] FLAG: --manifest-url="" Jul 17 07:11:34.808960 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801593 2568 flags.go:64] FLAG: --manifest-url-header="" Jul 17 07:11:34.808960 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801597 2568 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 17 07:11:34.808960 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801600 2568 flags.go:64] FLAG: --max-open-files="1000000" Jul 17 07:11:34.808960 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801605 2568 flags.go:64] FLAG: --max-pods="110" Jul 17 07:11:34.808960 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801608 2568 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 17 07:11:34.808960 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801611 2568 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 17 07:11:34.808960 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801614 2568 flags.go:64] FLAG: --memory-manager-policy="None" Jul 17 07:11:34.808960 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801617 2568 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 17 07:11:34.808960 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801620 2568 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 17 07:11:34.808960 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801624 2568 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 17 07:11:34.808960 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801627 2568 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 17 07:11:34.808960 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801633 2568 flags.go:64] FLAG: --node-status-max-images="50" Jul 17 07:11:34.808960 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801636 2568 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 17 07:11:34.808960 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801639 2568 flags.go:64] FLAG: --oom-score-adj="-999" Jul 17 07:11:34.808960 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801642 2568 flags.go:64] FLAG: --pod-cidr="" Jul 17 07:11:34.808960 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801645 2568 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 17 07:11:34.808960 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801651 2568 flags.go:64] FLAG: --pod-manifest-path="" Jul 17 07:11:34.808960 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801654 2568 flags.go:64] FLAG: --pod-max-pids="-1" Jul 17 07:11:34.808960 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801657 2568 flags.go:64] FLAG: --pods-per-core="0" Jul 17 07:11:34.808960 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801660 2568 flags.go:64] FLAG: --port="10250" Jul 17 07:11:34.808960 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801662 2568 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 17 07:11:34.808960 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801668 2568 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-059705ef297247602" Jul 17 07:11:34.809575 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801671 2568 flags.go:64] FLAG: --qos-reserved="" Jul 17 07:11:34.809575 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801674 2568 flags.go:64] FLAG: --read-only-port="10255" Jul 17 07:11:34.809575 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801677 2568 flags.go:64] FLAG: --register-node="true" Jul 17 07:11:34.809575 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801680 2568 flags.go:64] FLAG: --register-with-taints="" Jul 17 07:11:34.809575 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801683 2568 flags.go:64] FLAG: --registry-burst="10" Jul 17 07:11:34.809575 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801686 2568 flags.go:64] FLAG: --registry-qps="5" Jul 17 07:11:34.809575 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801689 2568 flags.go:64] FLAG: --reserved-cpus="" Jul 17 07:11:34.809575 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801692 2568 flags.go:64] FLAG: --reserved-memory="" Jul 17 07:11:34.809575 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801695 2568 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 17 07:11:34.809575 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801698 2568 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 17 07:11:34.809575 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801701 2568 flags.go:64] FLAG: --rotate-certificates="false" Jul 17 07:11:34.809575 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801704 2568 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 17 07:11:34.809575 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801706 2568 flags.go:64] FLAG: --runonce="false" Jul 17 07:11:34.809575 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801709 2568 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 17 07:11:34.809575 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801713 2568 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 17 07:11:34.809575 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801716 2568 flags.go:64] FLAG: --seccomp-default="false" Jul 17 07:11:34.809575 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801719 2568 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 17 07:11:34.809575 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801722 2568 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 17 07:11:34.809575 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801725 2568 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 17 07:11:34.809575 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801728 2568 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 17 07:11:34.809575 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801731 2568 flags.go:64] FLAG: --storage-driver-password="root" Jul 17 07:11:34.809575 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801734 2568 flags.go:64] FLAG: --storage-driver-secure="false" Jul 17 07:11:34.809575 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801737 2568 flags.go:64] FLAG: --storage-driver-table="stats" Jul 17 07:11:34.809575 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801740 2568 flags.go:64] FLAG: --storage-driver-user="root" Jul 17 07:11:34.809575 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801742 2568 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 17 07:11:34.809575 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801745 2568 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 17 07:11:34.810215 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801749 2568 flags.go:64] FLAG: --system-cgroups="" Jul 17 07:11:34.810215 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801751 2568 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 17 07:11:34.810215 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801757 2568 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 17 07:11:34.810215 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801759 2568 flags.go:64] FLAG: --tls-cert-file="" Jul 17 07:11:34.810215 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801762 2568 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 17 07:11:34.810215 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801767 2568 flags.go:64] FLAG: --tls-min-version="" Jul 17 07:11:34.810215 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801770 2568 flags.go:64] FLAG: --tls-private-key-file="" Jul 17 07:11:34.810215 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801772 2568 flags.go:64] FLAG: --topology-manager-policy="none" Jul 17 07:11:34.810215 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801775 2568 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 17 07:11:34.810215 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801778 2568 flags.go:64] FLAG: --topology-manager-scope="container" Jul 17 07:11:34.810215 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801781 2568 flags.go:64] FLAG: --v="2" Jul 17 07:11:34.810215 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801784 2568 flags.go:64] FLAG: --version="false" Jul 17 07:11:34.810215 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801788 2568 flags.go:64] FLAG: --vmodule="" Jul 17 07:11:34.810215 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801792 2568 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 17 07:11:34.810215 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.801795 2568 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 17 07:11:34.810215 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801905 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 17 07:11:34.810215 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801909 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 17 07:11:34.810215 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801912 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 17 07:11:34.810215 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801915 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 17 07:11:34.810215 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801918 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 17 07:11:34.810215 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801921 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 17 07:11:34.810215 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801924 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 17 07:11:34.810215 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801927 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 17 07:11:34.810834 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801930 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 17 07:11:34.810834 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801932 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 17 07:11:34.810834 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801935 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 17 07:11:34.810834 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801937 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 17 07:11:34.810834 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801940 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 17 07:11:34.810834 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801942 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 17 07:11:34.810834 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801945 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 17 07:11:34.810834 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801948 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 17 07:11:34.810834 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801950 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 17 07:11:34.810834 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801953 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 17 07:11:34.810834 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801956 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 17 07:11:34.810834 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801958 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 17 07:11:34.810834 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801961 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 17 07:11:34.810834 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801963 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 17 07:11:34.810834 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801966 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 17 07:11:34.810834 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801968 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 17 07:11:34.810834 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801971 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 17 07:11:34.810834 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801973 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 17 07:11:34.810834 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801976 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 17 07:11:34.810834 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801978 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 17 07:11:34.811322 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801983 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 17 07:11:34.811322 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801986 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 17 07:11:34.811322 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801989 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 17 07:11:34.811322 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801992 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 17 07:11:34.811322 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801995 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 17 07:11:34.811322 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.801997 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 17 07:11:34.811322 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802000 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 17 07:11:34.811322 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802002 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 17 07:11:34.811322 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802005 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 17 07:11:34.811322 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802008 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 17 07:11:34.811322 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802011 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 17 07:11:34.811322 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802013 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 17 07:11:34.811322 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802016 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 17 07:11:34.811322 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802018 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 17 07:11:34.811322 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802021 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 17 07:11:34.811322 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802023 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 17 07:11:34.811322 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802026 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 17 07:11:34.811322 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802028 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 17 07:11:34.811322 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802031 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 17 07:11:34.811852 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802033 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 17 07:11:34.811852 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802036 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 17 07:11:34.811852 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802038 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 17 07:11:34.811852 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802041 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 17 07:11:34.811852 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802044 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 17 07:11:34.811852 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802046 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 17 07:11:34.811852 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802048 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 17 07:11:34.811852 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802051 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 17 07:11:34.811852 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802054 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 17 07:11:34.811852 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802056 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 17 07:11:34.811852 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802059 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 17 07:11:34.811852 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802061 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 17 07:11:34.811852 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802064 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 17 07:11:34.811852 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802067 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 17 07:11:34.811852 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802069 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 17 07:11:34.811852 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802072 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 17 07:11:34.811852 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802076 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 17 07:11:34.811852 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802079 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 17 07:11:34.811852 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802082 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 17 07:11:34.812311 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802084 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 17 07:11:34.812311 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802087 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 17 07:11:34.812311 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802089 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 17 07:11:34.812311 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802092 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 17 07:11:34.812311 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802094 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 17 07:11:34.812311 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802097 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 17 07:11:34.812311 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802099 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 17 07:11:34.812311 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802102 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 17 07:11:34.812311 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802104 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 17 07:11:34.812311 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802107 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 17 07:11:34.812311 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802109 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 17 07:11:34.812311 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802111 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 17 07:11:34.812311 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802114 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 17 07:11:34.812311 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802116 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 17 07:11:34.812311 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802119 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 17 07:11:34.812311 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802121 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 17 07:11:34.812311 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802124 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 17 07:11:34.812311 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802126 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 17 07:11:34.812311 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802129 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 17 07:11:34.812311 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802132 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 17 07:11:34.812818 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802134 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 17 07:11:34.812818 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802137 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 17 07:11:34.812818 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802141 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 17 07:11:34.812818 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802144 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 17 07:11:34.812818 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802147 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 17 07:11:34.812818 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802149 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 17 07:11:34.812818 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802152 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 17 07:11:34.812818 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802155 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 17 07:11:34.812818 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802157 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 17 07:11:34.812818 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802160 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 17 07:11:34.812818 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802163 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 17 07:11:34.812818 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802165 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 17 07:11:34.812818 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.802167 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 17 07:11:34.812818 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.803088 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 17 07:11:34.812818 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.810496 2568 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 17 07:11:34.813190 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.810510 2568 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 17 07:11:34.813190 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.811417 2568 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 17 07:11:34.813190 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.811425 2568 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 17 07:11:34.813190 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811481 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 17 07:11:34.813190 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811486 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 17 07:11:34.813190 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811489 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 17 07:11:34.813190 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811492 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 17 07:11:34.813190 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811496 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 17 07:11:34.813190 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811498 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 17 07:11:34.813190 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811501 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 17 07:11:34.813190 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811504 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 17 07:11:34.813190 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811507 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 17 07:11:34.813190 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811509 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 17 07:11:34.813190 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811512 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 17 07:11:34.813190 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811515 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 17 07:11:34.813190 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811518 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 17 07:11:34.813190 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811522 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 17 07:11:34.813190 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811526 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 17 07:11:34.813190 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811529 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 17 07:11:34.813660 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811532 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 17 07:11:34.813660 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811534 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 17 07:11:34.813660 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811537 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 17 07:11:34.813660 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811540 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 17 07:11:34.813660 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811542 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 17 07:11:34.813660 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811545 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 17 07:11:34.813660 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811548 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 17 07:11:34.813660 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811550 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 17 07:11:34.813660 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811553 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 17 07:11:34.813660 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811555 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 17 07:11:34.813660 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811557 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 17 07:11:34.813660 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811560 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 17 07:11:34.813660 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811562 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 17 07:11:34.813660 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811566 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 17 07:11:34.813660 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811569 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 17 07:11:34.813660 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811571 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 17 07:11:34.813660 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811574 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 17 07:11:34.813660 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811576 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 17 07:11:34.813660 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811579 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 17 07:11:34.813660 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811582 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 17 07:11:34.814142 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811584 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 17 07:11:34.814142 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811587 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 17 07:11:34.814142 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811590 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 17 07:11:34.814142 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811592 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 17 07:11:34.814142 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811594 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 17 07:11:34.814142 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811598 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 17 07:11:34.814142 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811602 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 17 07:11:34.814142 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811605 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 17 07:11:34.814142 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811607 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 17 07:11:34.814142 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811610 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 17 07:11:34.814142 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811613 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 17 07:11:34.814142 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811615 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 17 07:11:34.814142 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811617 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 17 07:11:34.814142 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811620 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 17 07:11:34.814142 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811622 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 17 07:11:34.814142 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811625 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 17 07:11:34.814142 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811627 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 17 07:11:34.814142 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811630 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 17 07:11:34.814142 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811633 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 17 07:11:34.814597 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811635 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 17 07:11:34.814597 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811637 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 17 07:11:34.814597 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811640 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 17 07:11:34.814597 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811642 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 17 07:11:34.814597 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811645 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 17 07:11:34.814597 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811648 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 17 07:11:34.814597 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811651 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 17 07:11:34.814597 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811654 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 17 07:11:34.814597 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811656 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 17 07:11:34.814597 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811659 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 17 07:11:34.814597 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811662 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 17 07:11:34.814597 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811664 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 17 07:11:34.814597 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811667 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 17 07:11:34.814597 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811670 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 17 07:11:34.814597 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811672 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 17 07:11:34.814597 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811675 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 17 07:11:34.814597 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811677 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 17 07:11:34.814597 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811681 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 17 07:11:34.814597 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811684 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 17 07:11:34.815106 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811687 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 17 07:11:34.815106 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811689 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 17 07:11:34.815106 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811691 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 17 07:11:34.815106 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811694 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 17 07:11:34.815106 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811697 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 17 07:11:34.815106 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811699 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 17 07:11:34.815106 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811702 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 17 07:11:34.815106 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811704 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 17 07:11:34.815106 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811707 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 17 07:11:34.815106 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811709 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 17 07:11:34.815106 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811712 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 17 07:11:34.815106 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811714 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 17 07:11:34.815106 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811717 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 17 07:11:34.815106 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811719 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 17 07:11:34.815106 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811722 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 17 07:11:34.815106 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811725 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 17 07:11:34.815106 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811728 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 17 07:11:34.815106 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811730 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 17 07:11:34.815106 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811733 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 17 07:11:34.815106 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811735 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 17 07:11:34.815604 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811738 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 17 07:11:34.815604 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811741 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 17 07:11:34.815604 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811743 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 17 07:11:34.815604 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811746 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 17 07:11:34.815604 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811748 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 17 07:11:34.815604 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.811753 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 17 07:11:34.815604 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811875 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 17 07:11:34.815604 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811880 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 17 07:11:34.815604 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811883 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 17 07:11:34.815604 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811886 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 17 07:11:34.815604 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811889 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 17 07:11:34.815604 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811891 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 17 07:11:34.815604 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811894 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 17 07:11:34.815604 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811897 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 17 07:11:34.815604 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811899 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 17 07:11:34.815986 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811902 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 17 07:11:34.815986 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811904 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 17 07:11:34.815986 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811907 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 17 07:11:34.815986 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811910 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 17 07:11:34.815986 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811913 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 17 07:11:34.815986 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811916 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 17 07:11:34.815986 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811918 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 17 07:11:34.815986 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811921 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 17 07:11:34.815986 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811924 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 17 07:11:34.815986 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811926 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 17 07:11:34.815986 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811929 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 17 07:11:34.815986 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811931 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 17 07:11:34.815986 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811934 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 17 07:11:34.815986 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811936 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 17 07:11:34.815986 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811939 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 17 07:11:34.815986 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811942 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 17 07:11:34.815986 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811945 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 17 07:11:34.815986 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811947 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 17 07:11:34.815986 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811950 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 17 07:11:34.815986 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811952 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 17 07:11:34.816489 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811955 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 17 07:11:34.816489 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811957 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 17 07:11:34.816489 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811960 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 17 07:11:34.816489 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811962 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 17 07:11:34.816489 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811966 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 17 07:11:34.816489 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811970 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 17 07:11:34.816489 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811973 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 17 07:11:34.816489 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811977 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 17 07:11:34.816489 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811980 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 17 07:11:34.816489 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811983 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 17 07:11:34.816489 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811986 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 17 07:11:34.816489 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811989 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 17 07:11:34.816489 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811991 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 17 07:11:34.816489 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811994 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 17 07:11:34.816489 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.811997 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 17 07:11:34.816489 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.812000 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 17 07:11:34.816489 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.812004 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 17 07:11:34.816489 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.812007 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 17 07:11:34.816489 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.812011 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 17 07:11:34.816954 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.812013 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 17 07:11:34.816954 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.812016 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 17 07:11:34.816954 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.812019 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 17 07:11:34.816954 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.812022 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 17 07:11:34.816954 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.812024 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 17 07:11:34.816954 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.812026 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 17 07:11:34.816954 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.812029 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 17 07:11:34.816954 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.812032 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 17 07:11:34.816954 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.812034 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 17 07:11:34.816954 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.812037 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 17 07:11:34.816954 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.812039 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 17 07:11:34.816954 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.812042 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 17 07:11:34.816954 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.812044 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 17 07:11:34.816954 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.812046 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 17 07:11:34.816954 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.812049 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 17 07:11:34.816954 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.812051 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 17 07:11:34.816954 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.812054 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 17 07:11:34.816954 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.812056 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 17 07:11:34.816954 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.812059 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 17 07:11:34.816954 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.812061 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 17 07:11:34.817452 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.812063 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 17 07:11:34.817452 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.812066 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 17 07:11:34.817452 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.812068 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 17 07:11:34.817452 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.812071 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 17 07:11:34.817452 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.812073 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 17 07:11:34.817452 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.812076 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 17 07:11:34.817452 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.812078 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 17 07:11:34.817452 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.812081 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 17 07:11:34.817452 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.812083 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 17 07:11:34.817452 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.812086 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 17 07:11:34.817452 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.812089 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 17 07:11:34.817452 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.812091 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 17 07:11:34.817452 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.812093 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 17 07:11:34.817452 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.812096 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 17 07:11:34.817452 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.812099 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 17 07:11:34.817452 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.812102 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 17 07:11:34.817452 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.812104 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 17 07:11:34.817452 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.812106 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 17 07:11:34.817452 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.812109 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 17 07:11:34.817452 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.812111 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 17 07:11:34.817934 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.812114 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 17 07:11:34.817934 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.812117 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 17 07:11:34.817934 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.812119 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 17 07:11:34.817934 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.812122 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 17 07:11:34.817934 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.812124 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 17 07:11:34.817934 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.812126 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 17 07:11:34.817934 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.812129 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 17 07:11:34.817934 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.812131 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 17 07:11:34.817934 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.812134 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 17 07:11:34.817934 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.812136 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 17 07:11:34.817934 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:34.812139 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 17 07:11:34.817934 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.812143 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 17 07:11:34.817934 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.812238 2568 server.go:962] "Client rotation is on, will bootstrap in background" Jul 17 07:11:34.817934 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.814657 2568 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 17 07:11:34.818277 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.816375 2568 server.go:1019] "Starting client certificate rotation" Jul 17 07:11:34.818277 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.816477 2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 17 07:11:34.818277 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.816510 2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 17 07:11:34.819173 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.819160 2568 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 17 07:11:34.821769 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.821754 2568 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 17 07:11:34.830745 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.830725 2568 log.go:25] "Validated CRI v1 runtime API" Jul 17 07:11:34.834067 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.834051 2568 log.go:25] "Validated CRI v1 image API" Jul 17 07:11:34.837552 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.837537 2568 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 17 07:11:34.840106 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.840083 2568 fs.go:135] Filesystem UUIDs: map[20590d96-3f65-461a-a741-b7fb1cad515f:/dev/nvme0n1p3 3bcad798-eeb1-4951-857c-d0a5f671a91b:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jul 17 07:11:34.840189 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.840107 2568 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 17 07:11:34.844656 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.844639 2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 17 07:11:34.846932 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.846823 2568 manager.go:216] Machine: {Timestamp:2026-07-17 07:11:34.845411199 +0000 UTC m=+0.368993401 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099037 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec27c4b575e66631ad85bd77213784e6 SystemUUID:ec27c4b5-75e6-6631-ad85-bd77213784e6 BootID:c4fe21ec-edee-4f41-b07f-74645e14afb3 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:a2:83:84:99:db Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:a2:83:84:99:db Speed:0 Mtu:9001} {Name:ovs-system MacAddress:4e:c0:89:b7:4c:b3 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 17 07:11:34.846932 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.846922 2568 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 17 07:11:34.847061 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.846999 2568 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 17 07:11:34.848970 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.848946 2568 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 17 07:11:34.849101 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.848973 2568 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-233.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 17 07:11:34.849837 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.849828 2568 topology_manager.go:138] "Creating topology manager with none policy" Jul 17 07:11:34.849870 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.849839 2568 container_manager_linux.go:309] "Creating device plugin manager" Jul 17 07:11:34.849870 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.849847 2568 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 17 07:11:34.849870 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.849860 2568 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 17 07:11:34.851797 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.851787 2568 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 17 07:11:34.856571 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.856560 2568 state_mem.go:36] "Initialized new in-memory state store" Jul 17 07:11:34.856674 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.856665 2568 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 17 07:11:34.856719 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.856693 2568 kubelet.go:486] "Attempting to sync node with API server" Jul 17 07:11:34.856719 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.856708 2568 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 17 07:11:34.856829 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.856721 2568 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 17 07:11:34.856829 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.856730 2568 kubelet.go:398] "Adding apiserver pod source" Jul 17 07:11:34.856829 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.856738 2568 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 17 07:11:34.857053 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.857043 2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 17 07:11:34.857087 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.857059 2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 17 07:11:34.859342 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.859326 2568 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 17 07:11:34.861945 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.861930 2568 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 17 07:11:34.862005 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.861970 2568 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 17 07:11:34.863406 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.863376 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 17 07:11:34.863496 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.863414 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 17 07:11:34.863496 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.863425 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 17 07:11:34.863496 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.863433 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 17 07:11:34.863496 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.863446 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 17 07:11:34.863496 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.863454 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 17 07:11:34.863496 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.863466 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 17 07:11:34.863496 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.863473 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 17 07:11:34.863496 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.863482 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 17 07:11:34.863496 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.863490 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 17 07:11:34.863735 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.863511 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 17 07:11:34.863735 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.863519 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 17 07:11:34.864427 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.864414 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 17 07:11:34.864427 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.864426 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 17 07:11:34.868220 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.868203 2568 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-233.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 17 07:11:34.868317 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.868293 2568 server.go:1268] "Started kubelet" Jul 17 07:11:34.868409 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.868352 2568 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 17 07:11:34.868477 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:34.868370 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 17 07:11:34.869100 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.868670 2568 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 17 07:11:34.869348 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.869325 2568 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 17 07:11:34.869464 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:34.869433 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-233.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 17 07:11:34.869743 ip-10-0-138-233 systemd[1]: Started Kubernetes Kubelet. Jul 17 07:11:34.870909 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.870871 2568 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 17 07:11:34.871460 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.871448 2568 server.go:310] "Adding debug handlers to kubelet server" Jul 17 07:11:34.878340 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:34.878305 2568 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 17 07:11:34.879685 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:34.878504 2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-233.ec2.internal.18c30232de31b77c default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-233.ec2.internal,UID:ip-10-0-138-233.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-233.ec2.internal,},FirstTimestamp:2026-07-17 07:11:34.868268924 +0000 UTC m=+0.391851104,LastTimestamp:2026-07-17 07:11:34.868268924 +0000 UTC m=+0.391851104,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-233.ec2.internal,}" Jul 17 07:11:34.885593 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.885576 2568 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 17 07:11:34.885687 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.885629 2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 17 07:11:34.885976 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:34.885656 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-233.ec2.internal\" not found" Jul 17 07:11:34.886313 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.886297 2568 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 17 07:11:34.886411 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.886299 2568 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 17 07:11:34.886411 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.886337 2568 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 17 07:11:34.886584 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.886460 2568 reconstruct.go:97] "Volume reconstruction finished" Jul 17 07:11:34.886584 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.886470 2568 reconciler.go:29] "Reconciler: start to sync state" Jul 17 07:11:34.886584 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.886513 2568 factory.go:55] Registering systemd factory Jul 17 07:11:34.886584 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.886528 2568 factory.go:223] Registration of the systemd container factory successfully Jul 17 07:11:34.886769 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.886715 2568 factory.go:153] Registering CRI-O factory Jul 17 07:11:34.886769 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.886729 2568 factory.go:223] Registration of the crio container factory successfully Jul 17 07:11:34.886872 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.886779 2568 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 17 07:11:34.886872 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.886814 2568 factory.go:103] Registering Raw factory Jul 17 07:11:34.886872 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.886831 2568 manager.go:1196] Started watching for new ooms in manager Jul 17 07:11:34.887327 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.887310 2568 manager.go:349] Starting recovery of all containers Jul 17 07:11:34.887954 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:34.887932 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 17 07:11:34.888060 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:34.888019 2568 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-233.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 17 07:11:34.895143 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.895119 2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hntbk" Jul 17 07:11:34.897728 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.897712 2568 manager.go:354] Recovery completed Jul 17 07:11:34.901455 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.901440 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 17 07:11:34.902941 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.902923 2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hntbk" Jul 17 07:11:34.904923 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.904907 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-233.ec2.internal" event="NodeHasSufficientMemory" Jul 17 07:11:34.904994 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.904945 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-233.ec2.internal" event="NodeHasNoDiskPressure" Jul 17 07:11:34.904994 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.904960 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-233.ec2.internal" event="NodeHasSufficientPID" Jul 17 07:11:34.905497 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.905483 2568 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 17 07:11:34.905497 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.905498 2568 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 17 07:11:34.905636 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.905515 2568 state_mem.go:36] "Initialized new in-memory state store" Jul 17 07:11:34.905636 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.905486 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 17 07:11:34.906172 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.906160 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-233.ec2.internal" event="NodeHasSufficientMemory" Jul 17 07:11:34.906172 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.906184 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-233.ec2.internal" event="NodeHasNoDiskPressure" Jul 17 07:11:34.906263 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.906198 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-233.ec2.internal" event="NodeHasSufficientPID" Jul 17 07:11:34.907686 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.907672 2568 policy_none.go:49] "None policy: Start" Jul 17 07:11:34.907752 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.907694 2568 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 17 07:11:34.907752 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.907706 2568 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 17 07:11:34.909794 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.909780 2568 policy_none.go:47] "Start" Jul 17 07:11:34.952679 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.952646 2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 17 07:11:34.964257 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.954049 2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 17 07:11:34.964257 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.954091 2568 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 17 07:11:34.964257 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.954114 2568 kubelet.go:2448] "Starting kubelet main sync loop" Jul 17 07:11:34.964257 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:34.954149 2568 kubelet.go:2472] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Jul 17 07:11:34.964257 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:34.955914 2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 17 07:11:34.986789 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:34.986769 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-233.ec2.internal\" not found" Jul 17 07:11:35.042745 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:35.042730 2568 manager.go:337] "Starting Device Plugin manager" Jul 17 07:11:35.042844 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:35.042828 2568 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 17 07:11:35.042844 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:35.042844 2568 server.go:87] "Starting device plugin registration server" Jul 17 07:11:35.043108 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:35.043096 2568 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 17 07:11:35.043143 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:35.043112 2568 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 17 07:11:35.043228 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:35.043199 2568 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 17 07:11:35.043351 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:35.043313 2568 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 17 07:11:35.043351 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:35.043321 2568 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 17 07:11:35.044006 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:35.043854 2568 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 17 07:11:35.044091 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:35.044027 2568 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-233.ec2.internal\" not found" Jul 17 07:11:35.054858 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:35.054841 2568 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-138-233.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-233.ec2.internal"] Jul 17 07:11:35.054936 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:35.054904 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 17 07:11:35.055671 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:35.055652 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-233.ec2.internal" event="NodeHasSufficientMemory" Jul 17 07:11:35.055719 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:35.055680 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-233.ec2.internal" event="NodeHasNoDiskPressure" Jul 17 07:11:35.055719 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:35.055692 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-233.ec2.internal" event="NodeHasSufficientPID" Jul 17 07:11:35.059553 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:35.058383 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 17 07:11:35.059553 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:35.058666 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-233.ec2.internal" Jul 17 07:11:35.059553 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:35.058721 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 17 07:11:35.060368 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:35.060350 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-233.ec2.internal" event="NodeHasSufficientMemory" Jul 17 07:11:35.060462 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:35.060378 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-233.ec2.internal" event="NodeHasNoDiskPressure" Jul 17 07:11:35.060462 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:35.060380 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-233.ec2.internal" event="NodeHasSufficientMemory" Jul 17 07:11:35.060462 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:35.060411 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-233.ec2.internal" event="NodeHasSufficientPID" Jul 17 07:11:35.060462 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:35.060418 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-233.ec2.internal" event="NodeHasNoDiskPressure" Jul 17 07:11:35.060462 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:35.060432 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-233.ec2.internal" event="NodeHasSufficientPID" Jul 17 07:11:35.062779 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:35.062765 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-233.ec2.internal" Jul 17 07:11:35.062839 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:35.062791 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 17 07:11:35.063496 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:35.063481 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-233.ec2.internal" event="NodeHasSufficientMemory" Jul 17 07:11:35.063564 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:35.063506 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-233.ec2.internal" event="NodeHasNoDiskPressure" Jul 17 07:11:35.063564 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:35.063521 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-233.ec2.internal" event="NodeHasSufficientPID" Jul 17 07:11:35.084931 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:35.084913 2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-233.ec2.internal\" not found" node="ip-10-0-138-233.ec2.internal" Jul 17 07:11:35.086929 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:35.086905 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/66a5663e408f43e5d633dcf4f079b8db-config\") pod \"kube-apiserver-proxy-ip-10-0-138-233.ec2.internal\" (UID: \"66a5663e408f43e5d633dcf4f079b8db\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-233.ec2.internal" Jul 17 07:11:35.087005 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:35.086933 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c6e4e077fdfb8959399f2ed04ffb3611-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-233.ec2.internal\" (UID: \"c6e4e077fdfb8959399f2ed04ffb3611\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-233.ec2.internal" Jul 17 07:11:35.087005 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:35.086965 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c6e4e077fdfb8959399f2ed04ffb3611-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-233.ec2.internal\" (UID: \"c6e4e077fdfb8959399f2ed04ffb3611\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-233.ec2.internal" Jul 17 07:11:35.088793 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:35.088767 2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-233.ec2.internal\" not found" node="ip-10-0-138-233.ec2.internal" Jul 17 07:11:35.092765 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:35.092751 2568 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-138-233.ec2.internal\" not found" node="ip-10-0-138-233.ec2.internal" Jul 17 07:11:35.143750 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:35.143731 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 17 07:11:35.144576 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:35.144562 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-233.ec2.internal" event="NodeHasSufficientMemory" Jul 17 07:11:35.144651 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:35.144587 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-233.ec2.internal" event="NodeHasNoDiskPressure" Jul 17 07:11:35.144651 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:35.144597 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-233.ec2.internal" event="NodeHasSufficientPID" Jul 17 07:11:35.144651 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:35.144626 2568 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-233.ec2.internal" Jul 17 07:11:35.154792 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:35.154777 2568 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-233.ec2.internal" Jul 17 07:11:35.154845 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:35.154806 2568 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-233.ec2.internal\": node \"ip-10-0-138-233.ec2.internal\" not found" Jul 17 07:11:35.187128 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:35.187106 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c6e4e077fdfb8959399f2ed04ffb3611-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-233.ec2.internal\" (UID: \"c6e4e077fdfb8959399f2ed04ffb3611\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-233.ec2.internal" Jul 17 07:11:35.187232 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:35.187137 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/66a5663e408f43e5d633dcf4f079b8db-config\") pod \"kube-apiserver-proxy-ip-10-0-138-233.ec2.internal\" (UID: \"66a5663e408f43e5d633dcf4f079b8db\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-233.ec2.internal" Jul 17 07:11:35.187232 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:35.187160 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c6e4e077fdfb8959399f2ed04ffb3611-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-233.ec2.internal\" (UID: \"c6e4e077fdfb8959399f2ed04ffb3611\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-233.ec2.internal" Jul 17 07:11:35.187232 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:35.187210 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c6e4e077fdfb8959399f2ed04ffb3611-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-233.ec2.internal\" (UID: \"c6e4e077fdfb8959399f2ed04ffb3611\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-233.ec2.internal" Jul 17 07:11:35.187346 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:35.187211 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c6e4e077fdfb8959399f2ed04ffb3611-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-233.ec2.internal\" (UID: \"c6e4e077fdfb8959399f2ed04ffb3611\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-233.ec2.internal" Jul 17 07:11:35.187346 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:35.187240 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/66a5663e408f43e5d633dcf4f079b8db-config\") pod \"kube-apiserver-proxy-ip-10-0-138-233.ec2.internal\" (UID: \"66a5663e408f43e5d633dcf4f079b8db\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-233.ec2.internal" Jul 17 07:11:35.187581 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:35.187565 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-233.ec2.internal\" not found" Jul 17 07:11:35.287810 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:35.287752 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-233.ec2.internal\" not found" Jul 17 07:11:35.388407 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:35.388366 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-233.ec2.internal\" not found" Jul 17 07:11:35.389493 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:35.389479 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-233.ec2.internal" Jul 17 07:11:35.393193 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:35.393177 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-233.ec2.internal" Jul 17 07:11:35.488878 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:35.488847 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-233.ec2.internal\" not found" Jul 17 07:11:35.589355 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:35.589286 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-233.ec2.internal\" not found" Jul 17 07:11:36.728587 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:36.728561 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-233.ec2.internal\" not found" Jul 17 07:11:36.750634 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.750613 2568 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 17 07:11:36.825453 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.825421 2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-233.ec2.internal" Jul 17 07:11:36.838256 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.838238 2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 17 07:11:36.840308 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.840294 2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-233.ec2.internal" Jul 17 07:11:36.845262 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.845245 2568 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 17 07:11:36.847147 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.847129 2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 17 07:11:36.854234 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.854220 2568 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 17 07:11:36.854356 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.854335 2568 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 17 07:11:36.854403 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.854352 2568 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 17 07:11:36.854403 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.854373 2568 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 17 07:11:36.896569 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.896513 2568 apiserver.go:52] "Watching apiserver" Jul 17 07:11:36.911723 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.911700 2568 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 17 07:11:36.914775 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.914750 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-4x4z6","kube-system/kube-apiserver-proxy-ip-10-0-138-233.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qs6t2","openshift-dns/node-resolver-t6g42","openshift-multus/multus-additional-cni-plugins-z2q9x","openshift-multus/network-metrics-daemon-rvx7b","openshift-network-operator/iptables-alerter-774vt","openshift-ovn-kubernetes/ovnkube-node-cpwz4","kube-system/konnectivity-agent-zwtfj","openshift-cluster-node-tuning-operator/tuned-nglnq","openshift-image-registry/node-ca-cw57w","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-233.ec2.internal","openshift-multus/multus-nfjfq","openshift-network-diagnostics/network-check-target-dtjs7"] Jul 17 07:11:36.917320 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.917303 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4x4z6" Jul 17 07:11:36.917404 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:36.917369 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4x4z6" podUID="2639225a-fbee-4ebd-b36f-7ad47f365e79" Jul 17 07:11:36.919546 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.919532 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qs6t2" Jul 17 07:11:36.921670 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.921652 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-t6g42" Jul 17 07:11:36.921877 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.921859 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-swdcr\"" Jul 17 07:11:36.921957 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.921942 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 17 07:11:36.922005 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.921963 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 17 07:11:36.922005 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.921943 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 17 07:11:36.923953 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.923932 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-z2q9x" Jul 17 07:11:36.924051 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.923988 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 17 07:11:36.924134 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.924107 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-8tdkk\"" Jul 17 07:11:36.924533 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.924520 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 17 07:11:36.924975 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.924961 2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 17 07:11:36.926068 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.926052 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-bfp9n\"" Jul 17 07:11:36.926131 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.926086 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 17 07:11:36.926184 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.926151 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 17 07:11:36.926184 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.926163 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 17 07:11:36.926276 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.926184 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 17 07:11:36.926358 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.926343 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rvx7b" Jul 17 07:11:36.926438 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:36.926415 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rvx7b" podUID="deb65c16-2d4c-40f0-aa9c-4399304b7222" Jul 17 07:11:36.926521 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.926471 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 17 07:11:36.928577 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.928562 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-774vt" Jul 17 07:11:36.929930 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.929913 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4fbcebdd-a629-4d9a-9ba1-39e350bfa786-os-release\") pod \"multus-additional-cni-plugins-z2q9x\" (UID: \"4fbcebdd-a629-4d9a-9ba1-39e350bfa786\") " pod="openshift-multus/multus-additional-cni-plugins-z2q9x" Jul 17 07:11:36.929990 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.929947 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4fbcebdd-a629-4d9a-9ba1-39e350bfa786-tuning-conf-dir\") pod \"multus-additional-cni-plugins-z2q9x\" (UID: \"4fbcebdd-a629-4d9a-9ba1-39e350bfa786\") " pod="openshift-multus/multus-additional-cni-plugins-z2q9x" Jul 17 07:11:36.930030 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.929976 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/deb65c16-2d4c-40f0-aa9c-4399304b7222-metrics-certs\") pod \"network-metrics-daemon-rvx7b\" (UID: \"deb65c16-2d4c-40f0-aa9c-4399304b7222\") " pod="openshift-multus/network-metrics-daemon-rvx7b" Jul 17 07:11:36.930072 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.930028 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2639225a-fbee-4ebd-b36f-7ad47f365e79-original-pull-secret\") pod \"global-pull-secret-syncer-4x4z6\" (UID: \"2639225a-fbee-4ebd-b36f-7ad47f365e79\") " pod="kube-system/global-pull-secret-syncer-4x4z6" Jul 17 07:11:36.930072 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.930066 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8c97ade1-4b91-446e-acca-12c828f10d6c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qs6t2\" (UID: \"8c97ade1-4b91-446e-acca-12c828f10d6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qs6t2" Jul 17 07:11:36.930161 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.930080 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6a12d3cc-3a1a-4241-92b6-7d6198bc33c3-hosts-file\") pod \"node-resolver-t6g42\" (UID: \"6a12d3cc-3a1a-4241-92b6-7d6198bc33c3\") " pod="openshift-dns/node-resolver-t6g42" Jul 17 07:11:36.930161 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.930094 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a12d3cc-3a1a-4241-92b6-7d6198bc33c3-tmp-dir\") pod \"node-resolver-t6g42\" (UID: \"6a12d3cc-3a1a-4241-92b6-7d6198bc33c3\") " pod="openshift-dns/node-resolver-t6g42" Jul 17 07:11:36.930161 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.930109 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4fbcebdd-a629-4d9a-9ba1-39e350bfa786-cni-binary-copy\") pod \"multus-additional-cni-plugins-z2q9x\" (UID: \"4fbcebdd-a629-4d9a-9ba1-39e350bfa786\") " pod="openshift-multus/multus-additional-cni-plugins-z2q9x" Jul 17 07:11:36.930161 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.930151 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2639225a-fbee-4ebd-b36f-7ad47f365e79-dbus\") pod \"global-pull-secret-syncer-4x4z6\" (UID: \"2639225a-fbee-4ebd-b36f-7ad47f365e79\") " pod="kube-system/global-pull-secret-syncer-4x4z6" Jul 17 07:11:36.930449 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.930183 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8c97ade1-4b91-446e-acca-12c828f10d6c-registration-dir\") pod \"aws-ebs-csi-driver-node-qs6t2\" (UID: \"8c97ade1-4b91-446e-acca-12c828f10d6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qs6t2" Jul 17 07:11:36.930514 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.930476 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8c97ade1-4b91-446e-acca-12c828f10d6c-device-dir\") pod \"aws-ebs-csi-driver-node-qs6t2\" (UID: \"8c97ade1-4b91-446e-acca-12c828f10d6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qs6t2" Jul 17 07:11:36.930586 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.930571 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8c97ade1-4b91-446e-acca-12c828f10d6c-sys-fs\") pod \"aws-ebs-csi-driver-node-qs6t2\" (UID: \"8c97ade1-4b91-446e-acca-12c828f10d6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qs6t2" Jul 17 07:11:36.930758 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.930644 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5dbb\" (UniqueName: \"kubernetes.io/projected/8c97ade1-4b91-446e-acca-12c828f10d6c-kube-api-access-x5dbb\") pod \"aws-ebs-csi-driver-node-qs6t2\" (UID: \"8c97ade1-4b91-446e-acca-12c828f10d6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qs6t2" Jul 17 07:11:36.930758 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.930680 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4fbcebdd-a629-4d9a-9ba1-39e350bfa786-system-cni-dir\") pod \"multus-additional-cni-plugins-z2q9x\" (UID: \"4fbcebdd-a629-4d9a-9ba1-39e350bfa786\") " pod="openshift-multus/multus-additional-cni-plugins-z2q9x" Jul 17 07:11:36.930758 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.930711 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nq4kb\" (UniqueName: \"kubernetes.io/projected/4fbcebdd-a629-4d9a-9ba1-39e350bfa786-kube-api-access-nq4kb\") pod \"multus-additional-cni-plugins-z2q9x\" (UID: \"4fbcebdd-a629-4d9a-9ba1-39e350bfa786\") " pod="openshift-multus/multus-additional-cni-plugins-z2q9x" Jul 17 07:11:36.930758 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.930737 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mndt\" (UniqueName: \"kubernetes.io/projected/6a12d3cc-3a1a-4241-92b6-7d6198bc33c3-kube-api-access-8mndt\") pod \"node-resolver-t6g42\" (UID: \"6a12d3cc-3a1a-4241-92b6-7d6198bc33c3\") " pod="openshift-dns/node-resolver-t6g42" Jul 17 07:11:36.931146 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.930768 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4fbcebdd-a629-4d9a-9ba1-39e350bfa786-cnibin\") pod \"multus-additional-cni-plugins-z2q9x\" (UID: \"4fbcebdd-a629-4d9a-9ba1-39e350bfa786\") " pod="openshift-multus/multus-additional-cni-plugins-z2q9x" Jul 17 07:11:36.931146 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.930930 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 17 07:11:36.931241 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.930826 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4fbcebdd-a629-4d9a-9ba1-39e350bfa786-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-z2q9x\" (UID: \"4fbcebdd-a629-4d9a-9ba1-39e350bfa786\") " pod="openshift-multus/multus-additional-cni-plugins-z2q9x" Jul 17 07:11:36.931241 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.931231 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-djd8s\"" Jul 17 07:11:36.931360 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.931232 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4fbcebdd-a629-4d9a-9ba1-39e350bfa786-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-z2q9x\" (UID: \"4fbcebdd-a629-4d9a-9ba1-39e350bfa786\") " pod="openshift-multus/multus-additional-cni-plugins-z2q9x" Jul 17 07:11:36.931496 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.931475 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvwb6\" (UniqueName: \"kubernetes.io/projected/deb65c16-2d4c-40f0-aa9c-4399304b7222-kube-api-access-qvwb6\") pod \"network-metrics-daemon-rvx7b\" (UID: \"deb65c16-2d4c-40f0-aa9c-4399304b7222\") " pod="openshift-multus/network-metrics-daemon-rvx7b" Jul 17 07:11:36.931576 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.931559 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 17 07:11:36.931651 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.931521 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2639225a-fbee-4ebd-b36f-7ad47f365e79-kubelet-config\") pod \"global-pull-secret-syncer-4x4z6\" (UID: \"2639225a-fbee-4ebd-b36f-7ad47f365e79\") " pod="kube-system/global-pull-secret-syncer-4x4z6" Jul 17 07:11:36.931803 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.931787 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8c97ade1-4b91-446e-acca-12c828f10d6c-socket-dir\") pod \"aws-ebs-csi-driver-node-qs6t2\" (UID: \"8c97ade1-4b91-446e-acca-12c828f10d6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qs6t2" Jul 17 07:11:36.932030 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.931899 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8c97ade1-4b91-446e-acca-12c828f10d6c-etc-selinux\") pod \"aws-ebs-csi-driver-node-qs6t2\" (UID: \"8c97ade1-4b91-446e-acca-12c828f10d6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qs6t2" Jul 17 07:11:36.932095 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.932024 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 17 07:11:36.933627 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.932468 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:36.935132 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.935117 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 17 07:11:36.935318 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.935303 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-lj8vv\"" Jul 17 07:11:36.935394 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.935380 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 17 07:11:36.935455 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.935433 2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 17 07:11:36.935528 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.935511 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 17 07:11:36.935576 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.935561 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 17 07:11:36.935647 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.935632 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zwtfj" Jul 17 07:11:36.936136 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.936120 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 17 07:11:36.936242 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.936166 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 17 07:11:36.937701 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.937672 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 17 07:11:36.937833 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.937680 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 17 07:11:36.937945 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.937926 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-rw5kv\"" Jul 17 07:11:36.938063 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.938048 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-nglnq" Jul 17 07:11:36.939999 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.939985 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 17 07:11:36.940060 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.939989 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-d4zbc\"" Jul 17 07:11:36.940091 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.940064 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 17 07:11:36.940489 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.940472 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-cw57w" Jul 17 07:11:36.942356 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.942341 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 17 07:11:36.942452 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.942438 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 17 07:11:36.942567 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.942554 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 17 07:11:36.942643 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.942631 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-ppjnf\"" Jul 17 07:11:36.943279 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.943265 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nfjfq" Jul 17 07:11:36.945301 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.945268 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 17 07:11:36.945359 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.945314 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-b7xqq\"" Jul 17 07:11:36.945577 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.945556 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dtjs7" Jul 17 07:11:36.945651 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:36.945604 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dtjs7" podUID="ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376" Jul 17 07:11:36.948115 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.948079 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-16 07:06:35 +0000 UTC" deadline="2028-02-24 12:33:05.965205162 +0000 UTC" Jul 17 07:11:36.948115 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.948112 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14093h21m29.017094933s" Jul 17 07:11:36.967576 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.967557 2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-mqszq" Jul 17 07:11:36.975429 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:36.975413 2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-mqszq" Jul 17 07:11:37.025945 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.025928 2568 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 17 07:11:37.033055 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.033030 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1c44a3ae-4c83-4203-8b0b-116d6618accf-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cpwz4\" (UID: \"1c44a3ae-4c83-4203-8b0b-116d6618accf\") " pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.033159 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.033066 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1c44a3ae-4c83-4203-8b0b-116d6618accf-env-overrides\") pod \"ovnkube-node-cpwz4\" (UID: \"1c44a3ae-4c83-4203-8b0b-116d6618accf\") " pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.033159 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.033083 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/da1371ec-f502-49c4-85e7-91e52e5d6ce2-lib-modules\") pod \"tuned-nglnq\" (UID: \"da1371ec-f502-49c4-85e7-91e52e5d6ce2\") " pod="openshift-cluster-node-tuning-operator/tuned-nglnq" Jul 17 07:11:37.033159 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.033102 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4fbcebdd-a629-4d9a-9ba1-39e350bfa786-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-z2q9x\" (UID: \"4fbcebdd-a629-4d9a-9ba1-39e350bfa786\") " pod="openshift-multus/multus-additional-cni-plugins-z2q9x" Jul 17 07:11:37.033159 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.033118 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/01aeee2e-a151-4e27-9ca7-8e837d05cb13-host-slash\") pod \"iptables-alerter-774vt\" (UID: \"01aeee2e-a151-4e27-9ca7-8e837d05cb13\") " pod="openshift-network-operator/iptables-alerter-774vt" Jul 17 07:11:37.033159 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.033132 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/98ab979e-fe39-451d-b8d4-0098ea50c37f-agent-certs\") pod \"konnectivity-agent-zwtfj\" (UID: \"98ab979e-fe39-451d-b8d4-0098ea50c37f\") " pod="kube-system/konnectivity-agent-zwtfj" Jul 17 07:11:37.033396 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.033181 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/da1371ec-f502-49c4-85e7-91e52e5d6ce2-run\") pod \"tuned-nglnq\" (UID: \"da1371ec-f502-49c4-85e7-91e52e5d6ce2\") " pod="openshift-cluster-node-tuning-operator/tuned-nglnq" Jul 17 07:11:37.033396 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.033216 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/19b25751-5376-4627-9965-d850af34a46d-multus-socket-dir-parent\") pod \"multus-nfjfq\" (UID: \"19b25751-5376-4627-9965-d850af34a46d\") " pod="openshift-multus/multus-nfjfq" Jul 17 07:11:37.033396 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.033247 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/deb65c16-2d4c-40f0-aa9c-4399304b7222-metrics-certs\") pod \"network-metrics-daemon-rvx7b\" (UID: \"deb65c16-2d4c-40f0-aa9c-4399304b7222\") " pod="openshift-multus/network-metrics-daemon-rvx7b" Jul 17 07:11:37.033396 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.033273 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2639225a-fbee-4ebd-b36f-7ad47f365e79-original-pull-secret\") pod \"global-pull-secret-syncer-4x4z6\" (UID: \"2639225a-fbee-4ebd-b36f-7ad47f365e79\") " pod="kube-system/global-pull-secret-syncer-4x4z6" Jul 17 07:11:37.033396 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.033320 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6a12d3cc-3a1a-4241-92b6-7d6198bc33c3-hosts-file\") pod \"node-resolver-t6g42\" (UID: \"6a12d3cc-3a1a-4241-92b6-7d6198bc33c3\") " pod="openshift-dns/node-resolver-t6g42" Jul 17 07:11:37.033396 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.033345 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1c44a3ae-4c83-4203-8b0b-116d6618accf-host-run-netns\") pod \"ovnkube-node-cpwz4\" (UID: \"1c44a3ae-4c83-4203-8b0b-116d6618accf\") " pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.033396 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.033370 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4fbcebdd-a629-4d9a-9ba1-39e350bfa786-cni-binary-copy\") pod \"multus-additional-cni-plugins-z2q9x\" (UID: \"4fbcebdd-a629-4d9a-9ba1-39e350bfa786\") " pod="openshift-multus/multus-additional-cni-plugins-z2q9x" Jul 17 07:11:37.033396 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:37.033380 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 17 07:11:37.033396 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.033394 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2639225a-fbee-4ebd-b36f-7ad47f365e79-dbus\") pod \"global-pull-secret-syncer-4x4z6\" (UID: \"2639225a-fbee-4ebd-b36f-7ad47f365e79\") " pod="kube-system/global-pull-secret-syncer-4x4z6" Jul 17 07:11:37.033861 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:37.033461 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2639225a-fbee-4ebd-b36f-7ad47f365e79-original-pull-secret podName:2639225a-fbee-4ebd-b36f-7ad47f365e79 nodeName:}" failed. No retries permitted until 2026-07-17 07:11:37.533436888 +0000 UTC m=+2.018125844 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2639225a-fbee-4ebd-b36f-7ad47f365e79-original-pull-secret") pod "global-pull-secret-syncer-4x4z6" (UID: "2639225a-fbee-4ebd-b36f-7ad47f365e79") : object "kube-system"/"original-pull-secret" not registered Jul 17 07:11:37.033861 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.033457 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6a12d3cc-3a1a-4241-92b6-7d6198bc33c3-hosts-file\") pod \"node-resolver-t6g42\" (UID: \"6a12d3cc-3a1a-4241-92b6-7d6198bc33c3\") " pod="openshift-dns/node-resolver-t6g42" Jul 17 07:11:37.033861 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:37.033383 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 07:11:37.033861 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:37.033518 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/deb65c16-2d4c-40f0-aa9c-4399304b7222-metrics-certs podName:deb65c16-2d4c-40f0-aa9c-4399304b7222 nodeName:}" failed. No retries permitted until 2026-07-17 07:11:37.533506658 +0000 UTC m=+2.018195611 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/deb65c16-2d4c-40f0-aa9c-4399304b7222-metrics-certs") pod "network-metrics-daemon-rvx7b" (UID: "deb65c16-2d4c-40f0-aa9c-4399304b7222") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 07:11:37.033861 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.033479 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/01aeee2e-a151-4e27-9ca7-8e837d05cb13-iptables-alerter-script\") pod \"iptables-alerter-774vt\" (UID: \"01aeee2e-a151-4e27-9ca7-8e837d05cb13\") " pod="openshift-network-operator/iptables-alerter-774vt" Jul 17 07:11:37.033861 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.033549 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1c44a3ae-4c83-4203-8b0b-116d6618accf-ovn-node-metrics-cert\") pod \"ovnkube-node-cpwz4\" (UID: \"1c44a3ae-4c83-4203-8b0b-116d6618accf\") " pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.033861 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.033577 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/da1371ec-f502-49c4-85e7-91e52e5d6ce2-etc-sysctl-conf\") pod \"tuned-nglnq\" (UID: \"da1371ec-f502-49c4-85e7-91e52e5d6ce2\") " pod="openshift-cluster-node-tuning-operator/tuned-nglnq" Jul 17 07:11:37.033861 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.033597 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2639225a-fbee-4ebd-b36f-7ad47f365e79-dbus\") pod \"global-pull-secret-syncer-4x4z6\" (UID: \"2639225a-fbee-4ebd-b36f-7ad47f365e79\") " pod="kube-system/global-pull-secret-syncer-4x4z6" Jul 17 07:11:37.033861 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.033600 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/19b25751-5376-4627-9965-d850af34a46d-system-cni-dir\") pod \"multus-nfjfq\" (UID: \"19b25751-5376-4627-9965-d850af34a46d\") " pod="openshift-multus/multus-nfjfq" Jul 17 07:11:37.033861 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.033634 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkthr\" (UniqueName: \"kubernetes.io/projected/01aeee2e-a151-4e27-9ca7-8e837d05cb13-kube-api-access-tkthr\") pod \"iptables-alerter-774vt\" (UID: \"01aeee2e-a151-4e27-9ca7-8e837d05cb13\") " pod="openshift-network-operator/iptables-alerter-774vt" Jul 17 07:11:37.033861 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.033659 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1c44a3ae-4c83-4203-8b0b-116d6618accf-run-systemd\") pod \"ovnkube-node-cpwz4\" (UID: \"1c44a3ae-4c83-4203-8b0b-116d6618accf\") " pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.033861 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.033677 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4fbcebdd-a629-4d9a-9ba1-39e350bfa786-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-z2q9x\" (UID: \"4fbcebdd-a629-4d9a-9ba1-39e350bfa786\") " pod="openshift-multus/multus-additional-cni-plugins-z2q9x" Jul 17 07:11:37.033861 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.033684 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1c44a3ae-4c83-4203-8b0b-116d6618accf-log-socket\") pod \"ovnkube-node-cpwz4\" (UID: \"1c44a3ae-4c83-4203-8b0b-116d6618accf\") " pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.033861 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.033755 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1c44a3ae-4c83-4203-8b0b-116d6618accf-host-run-ovn-kubernetes\") pod \"ovnkube-node-cpwz4\" (UID: \"1c44a3ae-4c83-4203-8b0b-116d6618accf\") " pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.033861 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.033781 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3bc6540c-1f26-4a5e-9f66-e3480272699d-host\") pod \"node-ca-cw57w\" (UID: \"3bc6540c-1f26-4a5e-9f66-e3480272699d\") " pod="openshift-image-registry/node-ca-cw57w" Jul 17 07:11:37.033861 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.033796 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/19b25751-5376-4627-9965-d850af34a46d-cni-binary-copy\") pod \"multus-nfjfq\" (UID: \"19b25751-5376-4627-9965-d850af34a46d\") " pod="openshift-multus/multus-nfjfq" Jul 17 07:11:37.034650 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.033810 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jcw5\" (UniqueName: \"kubernetes.io/projected/19b25751-5376-4627-9965-d850af34a46d-kube-api-access-4jcw5\") pod \"multus-nfjfq\" (UID: \"19b25751-5376-4627-9965-d850af34a46d\") " pod="openshift-multus/multus-nfjfq" Jul 17 07:11:37.034650 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.033837 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4fbcebdd-a629-4d9a-9ba1-39e350bfa786-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-z2q9x\" (UID: \"4fbcebdd-a629-4d9a-9ba1-39e350bfa786\") " pod="openshift-multus/multus-additional-cni-plugins-z2q9x" Jul 17 07:11:37.034650 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.033861 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8c97ade1-4b91-446e-acca-12c828f10d6c-etc-selinux\") pod \"aws-ebs-csi-driver-node-qs6t2\" (UID: \"8c97ade1-4b91-446e-acca-12c828f10d6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qs6t2" Jul 17 07:11:37.034650 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.033884 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1c44a3ae-4c83-4203-8b0b-116d6618accf-run-openvswitch\") pod \"ovnkube-node-cpwz4\" (UID: \"1c44a3ae-4c83-4203-8b0b-116d6618accf\") " pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.034650 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.033907 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4fbcebdd-a629-4d9a-9ba1-39e350bfa786-tuning-conf-dir\") pod \"multus-additional-cni-plugins-z2q9x\" (UID: \"4fbcebdd-a629-4d9a-9ba1-39e350bfa786\") " pod="openshift-multus/multus-additional-cni-plugins-z2q9x" Jul 17 07:11:37.034650 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.033927 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1c44a3ae-4c83-4203-8b0b-116d6618accf-host-cni-bin\") pod \"ovnkube-node-cpwz4\" (UID: \"1c44a3ae-4c83-4203-8b0b-116d6618accf\") " pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.034650 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.033959 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8c97ade1-4b91-446e-acca-12c828f10d6c-etc-selinux\") pod \"aws-ebs-csi-driver-node-qs6t2\" (UID: \"8c97ade1-4b91-446e-acca-12c828f10d6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qs6t2" Jul 17 07:11:37.034650 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.033969 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4fbcebdd-a629-4d9a-9ba1-39e350bfa786-cni-binary-copy\") pod \"multus-additional-cni-plugins-z2q9x\" (UID: \"4fbcebdd-a629-4d9a-9ba1-39e350bfa786\") " pod="openshift-multus/multus-additional-cni-plugins-z2q9x" Jul 17 07:11:37.034650 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.033999 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/da1371ec-f502-49c4-85e7-91e52e5d6ce2-etc-sysctl-d\") pod \"tuned-nglnq\" (UID: \"da1371ec-f502-49c4-85e7-91e52e5d6ce2\") " pod="openshift-cluster-node-tuning-operator/tuned-nglnq" Jul 17 07:11:37.034650 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.034037 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/19b25751-5376-4627-9965-d850af34a46d-cnibin\") pod \"multus-nfjfq\" (UID: \"19b25751-5376-4627-9965-d850af34a46d\") " pod="openshift-multus/multus-nfjfq" Jul 17 07:11:37.034650 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.034060 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/19b25751-5376-4627-9965-d850af34a46d-multus-daemon-config\") pod \"multus-nfjfq\" (UID: \"19b25751-5376-4627-9965-d850af34a46d\") " pod="openshift-multus/multus-nfjfq" Jul 17 07:11:37.034650 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.034082 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c44a3ae-4c83-4203-8b0b-116d6618accf-host-kubelet\") pod \"ovnkube-node-cpwz4\" (UID: \"1c44a3ae-4c83-4203-8b0b-116d6618accf\") " pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.034650 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.034084 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4fbcebdd-a629-4d9a-9ba1-39e350bfa786-tuning-conf-dir\") pod \"multus-additional-cni-plugins-z2q9x\" (UID: \"4fbcebdd-a629-4d9a-9ba1-39e350bfa786\") " pod="openshift-multus/multus-additional-cni-plugins-z2q9x" Jul 17 07:11:37.034650 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.034106 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1c44a3ae-4c83-4203-8b0b-116d6618accf-systemd-units\") pod \"ovnkube-node-cpwz4\" (UID: \"1c44a3ae-4c83-4203-8b0b-116d6618accf\") " pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.034650 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.034133 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1c44a3ae-4c83-4203-8b0b-116d6618accf-ovnkube-script-lib\") pod \"ovnkube-node-cpwz4\" (UID: \"1c44a3ae-4c83-4203-8b0b-116d6618accf\") " pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.034650 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.034156 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/da1371ec-f502-49c4-85e7-91e52e5d6ce2-var-lib-kubelet\") pod \"tuned-nglnq\" (UID: \"da1371ec-f502-49c4-85e7-91e52e5d6ce2\") " pod="openshift-cluster-node-tuning-operator/tuned-nglnq" Jul 17 07:11:37.035294 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.034180 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wc2r7\" (UniqueName: \"kubernetes.io/projected/da1371ec-f502-49c4-85e7-91e52e5d6ce2-kube-api-access-wc2r7\") pod \"tuned-nglnq\" (UID: \"da1371ec-f502-49c4-85e7-91e52e5d6ce2\") " pod="openshift-cluster-node-tuning-operator/tuned-nglnq" Jul 17 07:11:37.035294 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.034215 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/19b25751-5376-4627-9965-d850af34a46d-etc-kubernetes\") pod \"multus-nfjfq\" (UID: \"19b25751-5376-4627-9965-d850af34a46d\") " pod="openshift-multus/multus-nfjfq" Jul 17 07:11:37.035294 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.034245 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4fbcebdd-a629-4d9a-9ba1-39e350bfa786-system-cni-dir\") pod \"multus-additional-cni-plugins-z2q9x\" (UID: \"4fbcebdd-a629-4d9a-9ba1-39e350bfa786\") " pod="openshift-multus/multus-additional-cni-plugins-z2q9x" Jul 17 07:11:37.035294 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.034270 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1c44a3ae-4c83-4203-8b0b-116d6618accf-host-slash\") pod \"ovnkube-node-cpwz4\" (UID: \"1c44a3ae-4c83-4203-8b0b-116d6618accf\") " pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.035294 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.034311 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1c44a3ae-4c83-4203-8b0b-116d6618accf-run-ovn\") pod \"ovnkube-node-cpwz4\" (UID: \"1c44a3ae-4c83-4203-8b0b-116d6618accf\") " pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.035294 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.034328 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4fbcebdd-a629-4d9a-9ba1-39e350bfa786-system-cni-dir\") pod \"multus-additional-cni-plugins-z2q9x\" (UID: \"4fbcebdd-a629-4d9a-9ba1-39e350bfa786\") " pod="openshift-multus/multus-additional-cni-plugins-z2q9x" Jul 17 07:11:37.035294 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.034363 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6t6zk\" (UniqueName: \"kubernetes.io/projected/1c44a3ae-4c83-4203-8b0b-116d6618accf-kube-api-access-6t6zk\") pod \"ovnkube-node-cpwz4\" (UID: \"1c44a3ae-4c83-4203-8b0b-116d6618accf\") " pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.035294 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.034390 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/da1371ec-f502-49c4-85e7-91e52e5d6ce2-etc-tuned\") pod \"tuned-nglnq\" (UID: \"da1371ec-f502-49c4-85e7-91e52e5d6ce2\") " pod="openshift-cluster-node-tuning-operator/tuned-nglnq" Jul 17 07:11:37.035294 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.034428 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/19b25751-5376-4627-9965-d850af34a46d-host-run-k8s-cni-cncf-io\") pod \"multus-nfjfq\" (UID: \"19b25751-5376-4627-9965-d850af34a46d\") " pod="openshift-multus/multus-nfjfq" Jul 17 07:11:37.035294 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.034450 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/19b25751-5376-4627-9965-d850af34a46d-host-run-netns\") pod \"multus-nfjfq\" (UID: \"19b25751-5376-4627-9965-d850af34a46d\") " pod="openshift-multus/multus-nfjfq" Jul 17 07:11:37.035294 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.034453 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4fbcebdd-a629-4d9a-9ba1-39e350bfa786-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-z2q9x\" (UID: \"4fbcebdd-a629-4d9a-9ba1-39e350bfa786\") " pod="openshift-multus/multus-additional-cni-plugins-z2q9x" Jul 17 07:11:37.035294 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.034482 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/19b25751-5376-4627-9965-d850af34a46d-host-run-multus-certs\") pod \"multus-nfjfq\" (UID: \"19b25751-5376-4627-9965-d850af34a46d\") " pod="openshift-multus/multus-nfjfq" Jul 17 07:11:37.035294 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.034536 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1c44a3ae-4c83-4203-8b0b-116d6618accf-ovnkube-config\") pod \"ovnkube-node-cpwz4\" (UID: \"1c44a3ae-4c83-4203-8b0b-116d6618accf\") " pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.035294 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.034571 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/19b25751-5376-4627-9965-d850af34a46d-host-var-lib-cni-bin\") pod \"multus-nfjfq\" (UID: \"19b25751-5376-4627-9965-d850af34a46d\") " pod="openshift-multus/multus-nfjfq" Jul 17 07:11:37.035294 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.034599 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1c44a3ae-4c83-4203-8b0b-116d6618accf-node-log\") pod \"ovnkube-node-cpwz4\" (UID: \"1c44a3ae-4c83-4203-8b0b-116d6618accf\") " pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.035294 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.034626 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wmqr\" (UniqueName: \"kubernetes.io/projected/ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376-kube-api-access-9wmqr\") pod \"network-check-target-dtjs7\" (UID: \"ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376\") " pod="openshift-network-diagnostics/network-check-target-dtjs7" Jul 17 07:11:37.035893 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.034652 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4fbcebdd-a629-4d9a-9ba1-39e350bfa786-os-release\") pod \"multus-additional-cni-plugins-z2q9x\" (UID: \"4fbcebdd-a629-4d9a-9ba1-39e350bfa786\") " pod="openshift-multus/multus-additional-cni-plugins-z2q9x" Jul 17 07:11:37.035893 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.034705 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8c97ade1-4b91-446e-acca-12c828f10d6c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qs6t2\" (UID: \"8c97ade1-4b91-446e-acca-12c828f10d6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qs6t2" Jul 17 07:11:37.035893 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.034730 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a12d3cc-3a1a-4241-92b6-7d6198bc33c3-tmp-dir\") pod \"node-resolver-t6g42\" (UID: \"6a12d3cc-3a1a-4241-92b6-7d6198bc33c3\") " pod="openshift-dns/node-resolver-t6g42" Jul 17 07:11:37.035893 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.034753 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/da1371ec-f502-49c4-85e7-91e52e5d6ce2-etc-kubernetes\") pod \"tuned-nglnq\" (UID: \"da1371ec-f502-49c4-85e7-91e52e5d6ce2\") " pod="openshift-cluster-node-tuning-operator/tuned-nglnq" Jul 17 07:11:37.035893 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.034773 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4fbcebdd-a629-4d9a-9ba1-39e350bfa786-os-release\") pod \"multus-additional-cni-plugins-z2q9x\" (UID: \"4fbcebdd-a629-4d9a-9ba1-39e350bfa786\") " pod="openshift-multus/multus-additional-cni-plugins-z2q9x" Jul 17 07:11:37.035893 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.034778 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8c97ade1-4b91-446e-acca-12c828f10d6c-registration-dir\") pod \"aws-ebs-csi-driver-node-qs6t2\" (UID: \"8c97ade1-4b91-446e-acca-12c828f10d6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qs6t2" Jul 17 07:11:37.035893 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.034780 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8c97ade1-4b91-446e-acca-12c828f10d6c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qs6t2\" (UID: \"8c97ade1-4b91-446e-acca-12c828f10d6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qs6t2" Jul 17 07:11:37.035893 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.034833 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5dbb\" (UniqueName: \"kubernetes.io/projected/8c97ade1-4b91-446e-acca-12c828f10d6c-kube-api-access-x5dbb\") pod \"aws-ebs-csi-driver-node-qs6t2\" (UID: \"8c97ade1-4b91-446e-acca-12c828f10d6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qs6t2" Jul 17 07:11:37.035893 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.034838 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8c97ade1-4b91-446e-acca-12c828f10d6c-registration-dir\") pod \"aws-ebs-csi-driver-node-qs6t2\" (UID: \"8c97ade1-4b91-446e-acca-12c828f10d6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qs6t2" Jul 17 07:11:37.035893 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.034857 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/da1371ec-f502-49c4-85e7-91e52e5d6ce2-sys\") pod \"tuned-nglnq\" (UID: \"da1371ec-f502-49c4-85e7-91e52e5d6ce2\") " pod="openshift-cluster-node-tuning-operator/tuned-nglnq" Jul 17 07:11:37.035893 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.034966 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/19b25751-5376-4627-9965-d850af34a46d-hostroot\") pod \"multus-nfjfq\" (UID: \"19b25751-5376-4627-9965-d850af34a46d\") " pod="openshift-multus/multus-nfjfq" Jul 17 07:11:37.035893 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.034999 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mndt\" (UniqueName: \"kubernetes.io/projected/6a12d3cc-3a1a-4241-92b6-7d6198bc33c3-kube-api-access-8mndt\") pod \"node-resolver-t6g42\" (UID: \"6a12d3cc-3a1a-4241-92b6-7d6198bc33c3\") " pod="openshift-dns/node-resolver-t6g42" Jul 17 07:11:37.035893 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.035029 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1c44a3ae-4c83-4203-8b0b-116d6618accf-var-lib-openvswitch\") pod \"ovnkube-node-cpwz4\" (UID: \"1c44a3ae-4c83-4203-8b0b-116d6618accf\") " pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.035893 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.035038 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a12d3cc-3a1a-4241-92b6-7d6198bc33c3-tmp-dir\") pod \"node-resolver-t6g42\" (UID: \"6a12d3cc-3a1a-4241-92b6-7d6198bc33c3\") " pod="openshift-dns/node-resolver-t6g42" Jul 17 07:11:37.035893 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.035053 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1c44a3ae-4c83-4203-8b0b-116d6618accf-etc-openvswitch\") pod \"ovnkube-node-cpwz4\" (UID: \"1c44a3ae-4c83-4203-8b0b-116d6618accf\") " pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.035893 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.035078 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/98ab979e-fe39-451d-b8d4-0098ea50c37f-konnectivity-ca\") pod \"konnectivity-agent-zwtfj\" (UID: \"98ab979e-fe39-451d-b8d4-0098ea50c37f\") " pod="kube-system/konnectivity-agent-zwtfj" Jul 17 07:11:37.035893 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.035101 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/da1371ec-f502-49c4-85e7-91e52e5d6ce2-etc-sysconfig\") pod \"tuned-nglnq\" (UID: \"da1371ec-f502-49c4-85e7-91e52e5d6ce2\") " pod="openshift-cluster-node-tuning-operator/tuned-nglnq" Jul 17 07:11:37.036608 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.035137 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/da1371ec-f502-49c4-85e7-91e52e5d6ce2-etc-systemd\") pod \"tuned-nglnq\" (UID: \"da1371ec-f502-49c4-85e7-91e52e5d6ce2\") " pod="openshift-cluster-node-tuning-operator/tuned-nglnq" Jul 17 07:11:37.036608 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.035178 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/da1371ec-f502-49c4-85e7-91e52e5d6ce2-tmp\") pod \"tuned-nglnq\" (UID: \"da1371ec-f502-49c4-85e7-91e52e5d6ce2\") " pod="openshift-cluster-node-tuning-operator/tuned-nglnq" Jul 17 07:11:37.036608 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.035211 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/19b25751-5376-4627-9965-d850af34a46d-host-var-lib-cni-multus\") pod \"multus-nfjfq\" (UID: \"19b25751-5376-4627-9965-d850af34a46d\") " pod="openshift-multus/multus-nfjfq" Jul 17 07:11:37.036608 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.035239 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4fbcebdd-a629-4d9a-9ba1-39e350bfa786-cnibin\") pod \"multus-additional-cni-plugins-z2q9x\" (UID: \"4fbcebdd-a629-4d9a-9ba1-39e350bfa786\") " pod="openshift-multus/multus-additional-cni-plugins-z2q9x" Jul 17 07:11:37.036608 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.035266 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvwb6\" (UniqueName: \"kubernetes.io/projected/deb65c16-2d4c-40f0-aa9c-4399304b7222-kube-api-access-qvwb6\") pod \"network-metrics-daemon-rvx7b\" (UID: \"deb65c16-2d4c-40f0-aa9c-4399304b7222\") " pod="openshift-multus/network-metrics-daemon-rvx7b" Jul 17 07:11:37.036608 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.035310 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4fbcebdd-a629-4d9a-9ba1-39e350bfa786-cnibin\") pod \"multus-additional-cni-plugins-z2q9x\" (UID: \"4fbcebdd-a629-4d9a-9ba1-39e350bfa786\") " pod="openshift-multus/multus-additional-cni-plugins-z2q9x" Jul 17 07:11:37.036608 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.035312 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2639225a-fbee-4ebd-b36f-7ad47f365e79-kubelet-config\") pod \"global-pull-secret-syncer-4x4z6\" (UID: \"2639225a-fbee-4ebd-b36f-7ad47f365e79\") " pod="kube-system/global-pull-secret-syncer-4x4z6" Jul 17 07:11:37.036608 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.035344 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8c97ade1-4b91-446e-acca-12c828f10d6c-socket-dir\") pod \"aws-ebs-csi-driver-node-qs6t2\" (UID: \"8c97ade1-4b91-446e-acca-12c828f10d6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qs6t2" Jul 17 07:11:37.036608 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.035360 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3bc6540c-1f26-4a5e-9f66-e3480272699d-serviceca\") pod \"node-ca-cw57w\" (UID: \"3bc6540c-1f26-4a5e-9f66-e3480272699d\") " pod="openshift-image-registry/node-ca-cw57w" Jul 17 07:11:37.036608 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.035373 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/da1371ec-f502-49c4-85e7-91e52e5d6ce2-host\") pod \"tuned-nglnq\" (UID: \"da1371ec-f502-49c4-85e7-91e52e5d6ce2\") " pod="openshift-cluster-node-tuning-operator/tuned-nglnq" Jul 17 07:11:37.036608 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.035387 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/19b25751-5376-4627-9965-d850af34a46d-os-release\") pod \"multus-nfjfq\" (UID: \"19b25751-5376-4627-9965-d850af34a46d\") " pod="openshift-multus/multus-nfjfq" Jul 17 07:11:37.036608 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.035399 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/19b25751-5376-4627-9965-d850af34a46d-host-var-lib-kubelet\") pod \"multus-nfjfq\" (UID: \"19b25751-5376-4627-9965-d850af34a46d\") " pod="openshift-multus/multus-nfjfq" Jul 17 07:11:37.036608 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.035413 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/da1371ec-f502-49c4-85e7-91e52e5d6ce2-etc-modprobe-d\") pod \"tuned-nglnq\" (UID: \"da1371ec-f502-49c4-85e7-91e52e5d6ce2\") " pod="openshift-cluster-node-tuning-operator/tuned-nglnq" Jul 17 07:11:37.036608 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.035445 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/19b25751-5376-4627-9965-d850af34a46d-multus-conf-dir\") pod \"multus-nfjfq\" (UID: \"19b25751-5376-4627-9965-d850af34a46d\") " pod="openshift-multus/multus-nfjfq" Jul 17 07:11:37.036608 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.035464 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2639225a-fbee-4ebd-b36f-7ad47f365e79-kubelet-config\") pod \"global-pull-secret-syncer-4x4z6\" (UID: \"2639225a-fbee-4ebd-b36f-7ad47f365e79\") " pod="kube-system/global-pull-secret-syncer-4x4z6" Jul 17 07:11:37.036608 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.035474 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8c97ade1-4b91-446e-acca-12c828f10d6c-device-dir\") pod \"aws-ebs-csi-driver-node-qs6t2\" (UID: \"8c97ade1-4b91-446e-acca-12c828f10d6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qs6t2" Jul 17 07:11:37.036608 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.035493 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8c97ade1-4b91-446e-acca-12c828f10d6c-sys-fs\") pod \"aws-ebs-csi-driver-node-qs6t2\" (UID: \"8c97ade1-4b91-446e-acca-12c828f10d6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qs6t2" Jul 17 07:11:37.037384 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.035508 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1c44a3ae-4c83-4203-8b0b-116d6618accf-host-cni-netd\") pod \"ovnkube-node-cpwz4\" (UID: \"1c44a3ae-4c83-4203-8b0b-116d6618accf\") " pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.037384 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.035531 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbj7x\" (UniqueName: \"kubernetes.io/projected/3bc6540c-1f26-4a5e-9f66-e3480272699d-kube-api-access-qbj7x\") pod \"node-ca-cw57w\" (UID: \"3bc6540c-1f26-4a5e-9f66-e3480272699d\") " pod="openshift-image-registry/node-ca-cw57w" Jul 17 07:11:37.037384 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.035540 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8c97ade1-4b91-446e-acca-12c828f10d6c-device-dir\") pod \"aws-ebs-csi-driver-node-qs6t2\" (UID: \"8c97ade1-4b91-446e-acca-12c828f10d6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qs6t2" Jul 17 07:11:37.037384 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.035553 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/19b25751-5376-4627-9965-d850af34a46d-multus-cni-dir\") pod \"multus-nfjfq\" (UID: \"19b25751-5376-4627-9965-d850af34a46d\") " pod="openshift-multus/multus-nfjfq" Jul 17 07:11:37.037384 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.035574 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nq4kb\" (UniqueName: \"kubernetes.io/projected/4fbcebdd-a629-4d9a-9ba1-39e350bfa786-kube-api-access-nq4kb\") pod \"multus-additional-cni-plugins-z2q9x\" (UID: \"4fbcebdd-a629-4d9a-9ba1-39e350bfa786\") " pod="openshift-multus/multus-additional-cni-plugins-z2q9x" Jul 17 07:11:37.037384 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.035589 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8c97ade1-4b91-446e-acca-12c828f10d6c-socket-dir\") pod \"aws-ebs-csi-driver-node-qs6t2\" (UID: \"8c97ade1-4b91-446e-acca-12c828f10d6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qs6t2" Jul 17 07:11:37.037384 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.035575 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8c97ade1-4b91-446e-acca-12c828f10d6c-sys-fs\") pod \"aws-ebs-csi-driver-node-qs6t2\" (UID: \"8c97ade1-4b91-446e-acca-12c828f10d6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qs6t2" Jul 17 07:11:37.041036 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.041017 2568 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 17 07:11:37.044344 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.044327 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5dbb\" (UniqueName: \"kubernetes.io/projected/8c97ade1-4b91-446e-acca-12c828f10d6c-kube-api-access-x5dbb\") pod \"aws-ebs-csi-driver-node-qs6t2\" (UID: \"8c97ade1-4b91-446e-acca-12c828f10d6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qs6t2" Jul 17 07:11:37.046886 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.046863 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nq4kb\" (UniqueName: \"kubernetes.io/projected/4fbcebdd-a629-4d9a-9ba1-39e350bfa786-kube-api-access-nq4kb\") pod \"multus-additional-cni-plugins-z2q9x\" (UID: \"4fbcebdd-a629-4d9a-9ba1-39e350bfa786\") " pod="openshift-multus/multus-additional-cni-plugins-z2q9x" Jul 17 07:11:37.047941 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.047917 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mndt\" (UniqueName: \"kubernetes.io/projected/6a12d3cc-3a1a-4241-92b6-7d6198bc33c3-kube-api-access-8mndt\") pod \"node-resolver-t6g42\" (UID: \"6a12d3cc-3a1a-4241-92b6-7d6198bc33c3\") " pod="openshift-dns/node-resolver-t6g42" Jul 17 07:11:37.048607 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.048588 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvwb6\" (UniqueName: \"kubernetes.io/projected/deb65c16-2d4c-40f0-aa9c-4399304b7222-kube-api-access-qvwb6\") pod \"network-metrics-daemon-rvx7b\" (UID: \"deb65c16-2d4c-40f0-aa9c-4399304b7222\") " pod="openshift-multus/network-metrics-daemon-rvx7b" Jul 17 07:11:37.136420 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.136399 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6t6zk\" (UniqueName: \"kubernetes.io/projected/1c44a3ae-4c83-4203-8b0b-116d6618accf-kube-api-access-6t6zk\") pod \"ovnkube-node-cpwz4\" (UID: \"1c44a3ae-4c83-4203-8b0b-116d6618accf\") " pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.136540 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.136428 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/da1371ec-f502-49c4-85e7-91e52e5d6ce2-etc-tuned\") pod \"tuned-nglnq\" (UID: \"da1371ec-f502-49c4-85e7-91e52e5d6ce2\") " pod="openshift-cluster-node-tuning-operator/tuned-nglnq" Jul 17 07:11:37.136540 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.136448 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/19b25751-5376-4627-9965-d850af34a46d-host-run-k8s-cni-cncf-io\") pod \"multus-nfjfq\" (UID: \"19b25751-5376-4627-9965-d850af34a46d\") " pod="openshift-multus/multus-nfjfq" Jul 17 07:11:37.136540 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.136511 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/19b25751-5376-4627-9965-d850af34a46d-host-run-k8s-cni-cncf-io\") pod \"multus-nfjfq\" (UID: \"19b25751-5376-4627-9965-d850af34a46d\") " pod="openshift-multus/multus-nfjfq" Jul 17 07:11:37.136701 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.136569 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/19b25751-5376-4627-9965-d850af34a46d-host-run-netns\") pod \"multus-nfjfq\" (UID: \"19b25751-5376-4627-9965-d850af34a46d\") " pod="openshift-multus/multus-nfjfq" Jul 17 07:11:37.136701 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.136607 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/19b25751-5376-4627-9965-d850af34a46d-host-run-multus-certs\") pod \"multus-nfjfq\" (UID: \"19b25751-5376-4627-9965-d850af34a46d\") " pod="openshift-multus/multus-nfjfq" Jul 17 07:11:37.136701 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.136653 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1c44a3ae-4c83-4203-8b0b-116d6618accf-ovnkube-config\") pod \"ovnkube-node-cpwz4\" (UID: \"1c44a3ae-4c83-4203-8b0b-116d6618accf\") " pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.136701 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.136689 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/19b25751-5376-4627-9965-d850af34a46d-host-run-netns\") pod \"multus-nfjfq\" (UID: \"19b25751-5376-4627-9965-d850af34a46d\") " pod="openshift-multus/multus-nfjfq" Jul 17 07:11:37.136894 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.136693 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/19b25751-5376-4627-9965-d850af34a46d-host-run-multus-certs\") pod \"multus-nfjfq\" (UID: \"19b25751-5376-4627-9965-d850af34a46d\") " pod="openshift-multus/multus-nfjfq" Jul 17 07:11:37.136894 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.136711 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/19b25751-5376-4627-9965-d850af34a46d-host-var-lib-cni-bin\") pod \"multus-nfjfq\" (UID: \"19b25751-5376-4627-9965-d850af34a46d\") " pod="openshift-multus/multus-nfjfq" Jul 17 07:11:37.136894 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.136739 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1c44a3ae-4c83-4203-8b0b-116d6618accf-node-log\") pod \"ovnkube-node-cpwz4\" (UID: \"1c44a3ae-4c83-4203-8b0b-116d6618accf\") " pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.136894 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.136769 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wmqr\" (UniqueName: \"kubernetes.io/projected/ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376-kube-api-access-9wmqr\") pod \"network-check-target-dtjs7\" (UID: \"ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376\") " pod="openshift-network-diagnostics/network-check-target-dtjs7" Jul 17 07:11:37.136894 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.136783 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/19b25751-5376-4627-9965-d850af34a46d-host-var-lib-cni-bin\") pod \"multus-nfjfq\" (UID: \"19b25751-5376-4627-9965-d850af34a46d\") " pod="openshift-multus/multus-nfjfq" Jul 17 07:11:37.136894 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.136799 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1c44a3ae-4c83-4203-8b0b-116d6618accf-node-log\") pod \"ovnkube-node-cpwz4\" (UID: \"1c44a3ae-4c83-4203-8b0b-116d6618accf\") " pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.136894 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.136804 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/da1371ec-f502-49c4-85e7-91e52e5d6ce2-etc-kubernetes\") pod \"tuned-nglnq\" (UID: \"da1371ec-f502-49c4-85e7-91e52e5d6ce2\") " pod="openshift-cluster-node-tuning-operator/tuned-nglnq" Jul 17 07:11:37.136894 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.136843 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/da1371ec-f502-49c4-85e7-91e52e5d6ce2-sys\") pod \"tuned-nglnq\" (UID: \"da1371ec-f502-49c4-85e7-91e52e5d6ce2\") " pod="openshift-cluster-node-tuning-operator/tuned-nglnq" Jul 17 07:11:37.136894 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.136854 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/da1371ec-f502-49c4-85e7-91e52e5d6ce2-etc-kubernetes\") pod \"tuned-nglnq\" (UID: \"da1371ec-f502-49c4-85e7-91e52e5d6ce2\") " pod="openshift-cluster-node-tuning-operator/tuned-nglnq" Jul 17 07:11:37.136894 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.136865 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/19b25751-5376-4627-9965-d850af34a46d-hostroot\") pod \"multus-nfjfq\" (UID: \"19b25751-5376-4627-9965-d850af34a46d\") " pod="openshift-multus/multus-nfjfq" Jul 17 07:11:37.136894 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.136881 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1c44a3ae-4c83-4203-8b0b-116d6618accf-var-lib-openvswitch\") pod \"ovnkube-node-cpwz4\" (UID: \"1c44a3ae-4c83-4203-8b0b-116d6618accf\") " pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.136894 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.136897 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1c44a3ae-4c83-4203-8b0b-116d6618accf-etc-openvswitch\") pod \"ovnkube-node-cpwz4\" (UID: \"1c44a3ae-4c83-4203-8b0b-116d6618accf\") " pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.137470 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.136897 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/da1371ec-f502-49c4-85e7-91e52e5d6ce2-sys\") pod \"tuned-nglnq\" (UID: \"da1371ec-f502-49c4-85e7-91e52e5d6ce2\") " pod="openshift-cluster-node-tuning-operator/tuned-nglnq" Jul 17 07:11:37.137470 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.136924 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1c44a3ae-4c83-4203-8b0b-116d6618accf-var-lib-openvswitch\") pod \"ovnkube-node-cpwz4\" (UID: \"1c44a3ae-4c83-4203-8b0b-116d6618accf\") " pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.137470 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.136942 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1c44a3ae-4c83-4203-8b0b-116d6618accf-etc-openvswitch\") pod \"ovnkube-node-cpwz4\" (UID: \"1c44a3ae-4c83-4203-8b0b-116d6618accf\") " pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.137470 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.136953 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/98ab979e-fe39-451d-b8d4-0098ea50c37f-konnectivity-ca\") pod \"konnectivity-agent-zwtfj\" (UID: \"98ab979e-fe39-451d-b8d4-0098ea50c37f\") " pod="kube-system/konnectivity-agent-zwtfj" Jul 17 07:11:37.137470 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.136971 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/19b25751-5376-4627-9965-d850af34a46d-hostroot\") pod \"multus-nfjfq\" (UID: \"19b25751-5376-4627-9965-d850af34a46d\") " pod="openshift-multus/multus-nfjfq" Jul 17 07:11:37.137470 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.136984 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/da1371ec-f502-49c4-85e7-91e52e5d6ce2-etc-sysconfig\") pod \"tuned-nglnq\" (UID: \"da1371ec-f502-49c4-85e7-91e52e5d6ce2\") " pod="openshift-cluster-node-tuning-operator/tuned-nglnq" Jul 17 07:11:37.137470 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.137009 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/da1371ec-f502-49c4-85e7-91e52e5d6ce2-etc-systemd\") pod \"tuned-nglnq\" (UID: \"da1371ec-f502-49c4-85e7-91e52e5d6ce2\") " pod="openshift-cluster-node-tuning-operator/tuned-nglnq" Jul 17 07:11:37.137470 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.137026 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/da1371ec-f502-49c4-85e7-91e52e5d6ce2-etc-sysconfig\") pod \"tuned-nglnq\" (UID: \"da1371ec-f502-49c4-85e7-91e52e5d6ce2\") " pod="openshift-cluster-node-tuning-operator/tuned-nglnq" Jul 17 07:11:37.137470 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.137032 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/da1371ec-f502-49c4-85e7-91e52e5d6ce2-tmp\") pod \"tuned-nglnq\" (UID: \"da1371ec-f502-49c4-85e7-91e52e5d6ce2\") " pod="openshift-cluster-node-tuning-operator/tuned-nglnq" Jul 17 07:11:37.137470 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.137075 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/da1371ec-f502-49c4-85e7-91e52e5d6ce2-etc-systemd\") pod \"tuned-nglnq\" (UID: \"da1371ec-f502-49c4-85e7-91e52e5d6ce2\") " pod="openshift-cluster-node-tuning-operator/tuned-nglnq" Jul 17 07:11:37.137470 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.137075 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/19b25751-5376-4627-9965-d850af34a46d-host-var-lib-cni-multus\") pod \"multus-nfjfq\" (UID: \"19b25751-5376-4627-9965-d850af34a46d\") " pod="openshift-multus/multus-nfjfq" Jul 17 07:11:37.137470 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.137104 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/19b25751-5376-4627-9965-d850af34a46d-host-var-lib-cni-multus\") pod \"multus-nfjfq\" (UID: \"19b25751-5376-4627-9965-d850af34a46d\") " pod="openshift-multus/multus-nfjfq" Jul 17 07:11:37.137470 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.137119 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3bc6540c-1f26-4a5e-9f66-e3480272699d-serviceca\") pod \"node-ca-cw57w\" (UID: \"3bc6540c-1f26-4a5e-9f66-e3480272699d\") " pod="openshift-image-registry/node-ca-cw57w" Jul 17 07:11:37.137470 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.137141 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/da1371ec-f502-49c4-85e7-91e52e5d6ce2-host\") pod \"tuned-nglnq\" (UID: \"da1371ec-f502-49c4-85e7-91e52e5d6ce2\") " pod="openshift-cluster-node-tuning-operator/tuned-nglnq" Jul 17 07:11:37.137470 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.137165 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/19b25751-5376-4627-9965-d850af34a46d-os-release\") pod \"multus-nfjfq\" (UID: \"19b25751-5376-4627-9965-d850af34a46d\") " pod="openshift-multus/multus-nfjfq" Jul 17 07:11:37.137470 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.137191 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/19b25751-5376-4627-9965-d850af34a46d-host-var-lib-kubelet\") pod \"multus-nfjfq\" (UID: \"19b25751-5376-4627-9965-d850af34a46d\") " pod="openshift-multus/multus-nfjfq" Jul 17 07:11:37.137470 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.137242 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/da1371ec-f502-49c4-85e7-91e52e5d6ce2-etc-modprobe-d\") pod \"tuned-nglnq\" (UID: \"da1371ec-f502-49c4-85e7-91e52e5d6ce2\") " pod="openshift-cluster-node-tuning-operator/tuned-nglnq" Jul 17 07:11:37.137470 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.137260 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1c44a3ae-4c83-4203-8b0b-116d6618accf-ovnkube-config\") pod \"ovnkube-node-cpwz4\" (UID: \"1c44a3ae-4c83-4203-8b0b-116d6618accf\") " pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.138248 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.137266 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/19b25751-5376-4627-9965-d850af34a46d-os-release\") pod \"multus-nfjfq\" (UID: \"19b25751-5376-4627-9965-d850af34a46d\") " pod="openshift-multus/multus-nfjfq" Jul 17 07:11:37.138248 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.137268 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/19b25751-5376-4627-9965-d850af34a46d-multus-conf-dir\") pod \"multus-nfjfq\" (UID: \"19b25751-5376-4627-9965-d850af34a46d\") " pod="openshift-multus/multus-nfjfq" Jul 17 07:11:37.138248 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.137332 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/19b25751-5376-4627-9965-d850af34a46d-host-var-lib-kubelet\") pod \"multus-nfjfq\" (UID: \"19b25751-5376-4627-9965-d850af34a46d\") " pod="openshift-multus/multus-nfjfq" Jul 17 07:11:37.138248 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.137383 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/19b25751-5376-4627-9965-d850af34a46d-multus-conf-dir\") pod \"multus-nfjfq\" (UID: \"19b25751-5376-4627-9965-d850af34a46d\") " pod="openshift-multus/multus-nfjfq" Jul 17 07:11:37.138248 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.137425 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1c44a3ae-4c83-4203-8b0b-116d6618accf-host-cni-netd\") pod \"ovnkube-node-cpwz4\" (UID: \"1c44a3ae-4c83-4203-8b0b-116d6618accf\") " pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.138248 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.137428 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/da1371ec-f502-49c4-85e7-91e52e5d6ce2-host\") pod \"tuned-nglnq\" (UID: \"da1371ec-f502-49c4-85e7-91e52e5d6ce2\") " pod="openshift-cluster-node-tuning-operator/tuned-nglnq" Jul 17 07:11:37.138248 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.137449 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbj7x\" (UniqueName: \"kubernetes.io/projected/3bc6540c-1f26-4a5e-9f66-e3480272699d-kube-api-access-qbj7x\") pod \"node-ca-cw57w\" (UID: \"3bc6540c-1f26-4a5e-9f66-e3480272699d\") " pod="openshift-image-registry/node-ca-cw57w" Jul 17 07:11:37.138248 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.137465 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/da1371ec-f502-49c4-85e7-91e52e5d6ce2-etc-modprobe-d\") pod \"tuned-nglnq\" (UID: \"da1371ec-f502-49c4-85e7-91e52e5d6ce2\") " pod="openshift-cluster-node-tuning-operator/tuned-nglnq" Jul 17 07:11:37.138248 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.137473 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/19b25751-5376-4627-9965-d850af34a46d-multus-cni-dir\") pod \"multus-nfjfq\" (UID: \"19b25751-5376-4627-9965-d850af34a46d\") " pod="openshift-multus/multus-nfjfq" Jul 17 07:11:37.138248 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.137479 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/98ab979e-fe39-451d-b8d4-0098ea50c37f-konnectivity-ca\") pod \"konnectivity-agent-zwtfj\" (UID: \"98ab979e-fe39-451d-b8d4-0098ea50c37f\") " pod="kube-system/konnectivity-agent-zwtfj" Jul 17 07:11:37.138248 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.137476 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1c44a3ae-4c83-4203-8b0b-116d6618accf-host-cni-netd\") pod \"ovnkube-node-cpwz4\" (UID: \"1c44a3ae-4c83-4203-8b0b-116d6618accf\") " pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.138248 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.137503 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1c44a3ae-4c83-4203-8b0b-116d6618accf-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cpwz4\" (UID: \"1c44a3ae-4c83-4203-8b0b-116d6618accf\") " pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.138248 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.137526 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/19b25751-5376-4627-9965-d850af34a46d-multus-cni-dir\") pod \"multus-nfjfq\" (UID: \"19b25751-5376-4627-9965-d850af34a46d\") " pod="openshift-multus/multus-nfjfq" Jul 17 07:11:37.138248 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.137528 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1c44a3ae-4c83-4203-8b0b-116d6618accf-env-overrides\") pod \"ovnkube-node-cpwz4\" (UID: \"1c44a3ae-4c83-4203-8b0b-116d6618accf\") " pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.138248 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.137928 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1c44a3ae-4c83-4203-8b0b-116d6618accf-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cpwz4\" (UID: \"1c44a3ae-4c83-4203-8b0b-116d6618accf\") " pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.138248 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.137971 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/da1371ec-f502-49c4-85e7-91e52e5d6ce2-lib-modules\") pod \"tuned-nglnq\" (UID: \"da1371ec-f502-49c4-85e7-91e52e5d6ce2\") " pod="openshift-cluster-node-tuning-operator/tuned-nglnq" Jul 17 07:11:37.138248 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.138032 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/01aeee2e-a151-4e27-9ca7-8e837d05cb13-host-slash\") pod \"iptables-alerter-774vt\" (UID: \"01aeee2e-a151-4e27-9ca7-8e837d05cb13\") " pod="openshift-network-operator/iptables-alerter-774vt" Jul 17 07:11:37.138248 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.138062 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/98ab979e-fe39-451d-b8d4-0098ea50c37f-agent-certs\") pod \"konnectivity-agent-zwtfj\" (UID: \"98ab979e-fe39-451d-b8d4-0098ea50c37f\") " pod="kube-system/konnectivity-agent-zwtfj" Jul 17 07:11:37.139064 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.138100 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/da1371ec-f502-49c4-85e7-91e52e5d6ce2-run\") pod \"tuned-nglnq\" (UID: \"da1371ec-f502-49c4-85e7-91e52e5d6ce2\") " pod="openshift-cluster-node-tuning-operator/tuned-nglnq" Jul 17 07:11:37.139064 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.138135 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/19b25751-5376-4627-9965-d850af34a46d-multus-socket-dir-parent\") pod \"multus-nfjfq\" (UID: \"19b25751-5376-4627-9965-d850af34a46d\") " pod="openshift-multus/multus-nfjfq" Jul 17 07:11:37.139064 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.137591 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3bc6540c-1f26-4a5e-9f66-e3480272699d-serviceca\") pod \"node-ca-cw57w\" (UID: \"3bc6540c-1f26-4a5e-9f66-e3480272699d\") " pod="openshift-image-registry/node-ca-cw57w" Jul 17 07:11:37.139064 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.138206 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1c44a3ae-4c83-4203-8b0b-116d6618accf-host-run-netns\") pod \"ovnkube-node-cpwz4\" (UID: \"1c44a3ae-4c83-4203-8b0b-116d6618accf\") " pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.139064 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.138246 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/01aeee2e-a151-4e27-9ca7-8e837d05cb13-iptables-alerter-script\") pod \"iptables-alerter-774vt\" (UID: \"01aeee2e-a151-4e27-9ca7-8e837d05cb13\") " pod="openshift-network-operator/iptables-alerter-774vt" Jul 17 07:11:37.139064 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.138275 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1c44a3ae-4c83-4203-8b0b-116d6618accf-ovn-node-metrics-cert\") pod \"ovnkube-node-cpwz4\" (UID: \"1c44a3ae-4c83-4203-8b0b-116d6618accf\") " pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.139064 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.138325 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/da1371ec-f502-49c4-85e7-91e52e5d6ce2-lib-modules\") pod \"tuned-nglnq\" (UID: \"da1371ec-f502-49c4-85e7-91e52e5d6ce2\") " pod="openshift-cluster-node-tuning-operator/tuned-nglnq" Jul 17 07:11:37.139064 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.138328 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/da1371ec-f502-49c4-85e7-91e52e5d6ce2-etc-sysctl-conf\") pod \"tuned-nglnq\" (UID: \"da1371ec-f502-49c4-85e7-91e52e5d6ce2\") " pod="openshift-cluster-node-tuning-operator/tuned-nglnq" Jul 17 07:11:37.139064 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.138379 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/19b25751-5376-4627-9965-d850af34a46d-system-cni-dir\") pod \"multus-nfjfq\" (UID: \"19b25751-5376-4627-9965-d850af34a46d\") " pod="openshift-multus/multus-nfjfq" Jul 17 07:11:37.139064 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.138414 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkthr\" (UniqueName: \"kubernetes.io/projected/01aeee2e-a151-4e27-9ca7-8e837d05cb13-kube-api-access-tkthr\") pod \"iptables-alerter-774vt\" (UID: \"01aeee2e-a151-4e27-9ca7-8e837d05cb13\") " pod="openshift-network-operator/iptables-alerter-774vt" Jul 17 07:11:37.139064 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.138464 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/da1371ec-f502-49c4-85e7-91e52e5d6ce2-etc-sysctl-conf\") pod \"tuned-nglnq\" (UID: \"da1371ec-f502-49c4-85e7-91e52e5d6ce2\") " pod="openshift-cluster-node-tuning-operator/tuned-nglnq" Jul 17 07:11:37.139064 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.138476 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1c44a3ae-4c83-4203-8b0b-116d6618accf-run-systemd\") pod \"ovnkube-node-cpwz4\" (UID: \"1c44a3ae-4c83-4203-8b0b-116d6618accf\") " pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.139064 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.138543 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/01aeee2e-a151-4e27-9ca7-8e837d05cb13-host-slash\") pod \"iptables-alerter-774vt\" (UID: \"01aeee2e-a151-4e27-9ca7-8e837d05cb13\") " pod="openshift-network-operator/iptables-alerter-774vt" Jul 17 07:11:37.139064 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.138574 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/19b25751-5376-4627-9965-d850af34a46d-multus-socket-dir-parent\") pod \"multus-nfjfq\" (UID: \"19b25751-5376-4627-9965-d850af34a46d\") " pod="openshift-multus/multus-nfjfq" Jul 17 07:11:37.139064 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.138591 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/da1371ec-f502-49c4-85e7-91e52e5d6ce2-run\") pod \"tuned-nglnq\" (UID: \"da1371ec-f502-49c4-85e7-91e52e5d6ce2\") " pod="openshift-cluster-node-tuning-operator/tuned-nglnq" Jul 17 07:11:37.139064 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.138626 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1c44a3ae-4c83-4203-8b0b-116d6618accf-host-run-netns\") pod \"ovnkube-node-cpwz4\" (UID: \"1c44a3ae-4c83-4203-8b0b-116d6618accf\") " pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.139064 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.138643 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/19b25751-5376-4627-9965-d850af34a46d-system-cni-dir\") pod \"multus-nfjfq\" (UID: \"19b25751-5376-4627-9965-d850af34a46d\") " pod="openshift-multus/multus-nfjfq" Jul 17 07:11:37.139064 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.138983 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1c44a3ae-4c83-4203-8b0b-116d6618accf-env-overrides\") pod \"ovnkube-node-cpwz4\" (UID: \"1c44a3ae-4c83-4203-8b0b-116d6618accf\") " pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.139858 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.139027 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/da1371ec-f502-49c4-85e7-91e52e5d6ce2-etc-tuned\") pod \"tuned-nglnq\" (UID: \"da1371ec-f502-49c4-85e7-91e52e5d6ce2\") " pod="openshift-cluster-node-tuning-operator/tuned-nglnq" Jul 17 07:11:37.139858 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.139161 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/01aeee2e-a151-4e27-9ca7-8e837d05cb13-iptables-alerter-script\") pod \"iptables-alerter-774vt\" (UID: \"01aeee2e-a151-4e27-9ca7-8e837d05cb13\") " pod="openshift-network-operator/iptables-alerter-774vt" Jul 17 07:11:37.139858 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.139161 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1c44a3ae-4c83-4203-8b0b-116d6618accf-run-systemd\") pod \"ovnkube-node-cpwz4\" (UID: \"1c44a3ae-4c83-4203-8b0b-116d6618accf\") " pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.139858 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.139219 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1c44a3ae-4c83-4203-8b0b-116d6618accf-log-socket\") pod \"ovnkube-node-cpwz4\" (UID: \"1c44a3ae-4c83-4203-8b0b-116d6618accf\") " pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.139858 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.139254 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1c44a3ae-4c83-4203-8b0b-116d6618accf-host-run-ovn-kubernetes\") pod \"ovnkube-node-cpwz4\" (UID: \"1c44a3ae-4c83-4203-8b0b-116d6618accf\") " pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.139858 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.139221 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/da1371ec-f502-49c4-85e7-91e52e5d6ce2-tmp\") pod \"tuned-nglnq\" (UID: \"da1371ec-f502-49c4-85e7-91e52e5d6ce2\") " pod="openshift-cluster-node-tuning-operator/tuned-nglnq" Jul 17 07:11:37.139858 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.139308 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3bc6540c-1f26-4a5e-9f66-e3480272699d-host\") pod \"node-ca-cw57w\" (UID: \"3bc6540c-1f26-4a5e-9f66-e3480272699d\") " pod="openshift-image-registry/node-ca-cw57w" Jul 17 07:11:37.139858 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.139372 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3bc6540c-1f26-4a5e-9f66-e3480272699d-host\") pod \"node-ca-cw57w\" (UID: \"3bc6540c-1f26-4a5e-9f66-e3480272699d\") " pod="openshift-image-registry/node-ca-cw57w" Jul 17 07:11:37.139858 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.139379 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/19b25751-5376-4627-9965-d850af34a46d-cni-binary-copy\") pod \"multus-nfjfq\" (UID: \"19b25751-5376-4627-9965-d850af34a46d\") " pod="openshift-multus/multus-nfjfq" Jul 17 07:11:37.139858 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.139391 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1c44a3ae-4c83-4203-8b0b-116d6618accf-log-socket\") pod \"ovnkube-node-cpwz4\" (UID: \"1c44a3ae-4c83-4203-8b0b-116d6618accf\") " pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.139858 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.139579 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1c44a3ae-4c83-4203-8b0b-116d6618accf-host-run-ovn-kubernetes\") pod \"ovnkube-node-cpwz4\" (UID: \"1c44a3ae-4c83-4203-8b0b-116d6618accf\") " pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.139858 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.139626 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jcw5\" (UniqueName: \"kubernetes.io/projected/19b25751-5376-4627-9965-d850af34a46d-kube-api-access-4jcw5\") pod \"multus-nfjfq\" (UID: \"19b25751-5376-4627-9965-d850af34a46d\") " pod="openshift-multus/multus-nfjfq" Jul 17 07:11:37.139858 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.139665 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1c44a3ae-4c83-4203-8b0b-116d6618accf-run-openvswitch\") pod \"ovnkube-node-cpwz4\" (UID: \"1c44a3ae-4c83-4203-8b0b-116d6618accf\") " pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.139858 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.139698 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1c44a3ae-4c83-4203-8b0b-116d6618accf-host-cni-bin\") pod \"ovnkube-node-cpwz4\" (UID: \"1c44a3ae-4c83-4203-8b0b-116d6618accf\") " pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.139858 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.139730 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/da1371ec-f502-49c4-85e7-91e52e5d6ce2-etc-sysctl-d\") pod \"tuned-nglnq\" (UID: \"da1371ec-f502-49c4-85e7-91e52e5d6ce2\") " pod="openshift-cluster-node-tuning-operator/tuned-nglnq" Jul 17 07:11:37.139858 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.139757 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/19b25751-5376-4627-9965-d850af34a46d-cnibin\") pod \"multus-nfjfq\" (UID: \"19b25751-5376-4627-9965-d850af34a46d\") " pod="openshift-multus/multus-nfjfq" Jul 17 07:11:37.139858 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.139789 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/19b25751-5376-4627-9965-d850af34a46d-multus-daemon-config\") pod \"multus-nfjfq\" (UID: \"19b25751-5376-4627-9965-d850af34a46d\") " pod="openshift-multus/multus-nfjfq" Jul 17 07:11:37.140682 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.139914 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c44a3ae-4c83-4203-8b0b-116d6618accf-host-kubelet\") pod \"ovnkube-node-cpwz4\" (UID: \"1c44a3ae-4c83-4203-8b0b-116d6618accf\") " pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.140682 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.139949 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1c44a3ae-4c83-4203-8b0b-116d6618accf-systemd-units\") pod \"ovnkube-node-cpwz4\" (UID: \"1c44a3ae-4c83-4203-8b0b-116d6618accf\") " pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.140682 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.139973 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/19b25751-5376-4627-9965-d850af34a46d-cni-binary-copy\") pod \"multus-nfjfq\" (UID: \"19b25751-5376-4627-9965-d850af34a46d\") " pod="openshift-multus/multus-nfjfq" Jul 17 07:11:37.140682 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.139981 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1c44a3ae-4c83-4203-8b0b-116d6618accf-ovnkube-script-lib\") pod \"ovnkube-node-cpwz4\" (UID: \"1c44a3ae-4c83-4203-8b0b-116d6618accf\") " pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.140682 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.140044 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/da1371ec-f502-49c4-85e7-91e52e5d6ce2-var-lib-kubelet\") pod \"tuned-nglnq\" (UID: \"da1371ec-f502-49c4-85e7-91e52e5d6ce2\") " pod="openshift-cluster-node-tuning-operator/tuned-nglnq" Jul 17 07:11:37.140682 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.140065 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1c44a3ae-4c83-4203-8b0b-116d6618accf-run-openvswitch\") pod \"ovnkube-node-cpwz4\" (UID: \"1c44a3ae-4c83-4203-8b0b-116d6618accf\") " pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.140682 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.140077 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wc2r7\" (UniqueName: \"kubernetes.io/projected/da1371ec-f502-49c4-85e7-91e52e5d6ce2-kube-api-access-wc2r7\") pod \"tuned-nglnq\" (UID: \"da1371ec-f502-49c4-85e7-91e52e5d6ce2\") " pod="openshift-cluster-node-tuning-operator/tuned-nglnq" Jul 17 07:11:37.140682 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.140103 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/da1371ec-f502-49c4-85e7-91e52e5d6ce2-etc-sysctl-d\") pod \"tuned-nglnq\" (UID: \"da1371ec-f502-49c4-85e7-91e52e5d6ce2\") " pod="openshift-cluster-node-tuning-operator/tuned-nglnq" Jul 17 07:11:37.140682 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.140126 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/19b25751-5376-4627-9965-d850af34a46d-etc-kubernetes\") pod \"multus-nfjfq\" (UID: \"19b25751-5376-4627-9965-d850af34a46d\") " pod="openshift-multus/multus-nfjfq" Jul 17 07:11:37.140682 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.140171 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1c44a3ae-4c83-4203-8b0b-116d6618accf-systemd-units\") pod \"ovnkube-node-cpwz4\" (UID: \"1c44a3ae-4c83-4203-8b0b-116d6618accf\") " pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.140682 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.140180 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/19b25751-5376-4627-9965-d850af34a46d-etc-kubernetes\") pod \"multus-nfjfq\" (UID: \"19b25751-5376-4627-9965-d850af34a46d\") " pod="openshift-multus/multus-nfjfq" Jul 17 07:11:37.140682 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.140224 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1c44a3ae-4c83-4203-8b0b-116d6618accf-host-slash\") pod \"ovnkube-node-cpwz4\" (UID: \"1c44a3ae-4c83-4203-8b0b-116d6618accf\") " pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.140682 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.140260 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1c44a3ae-4c83-4203-8b0b-116d6618accf-run-ovn\") pod \"ovnkube-node-cpwz4\" (UID: \"1c44a3ae-4c83-4203-8b0b-116d6618accf\") " pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.140682 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.140467 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1c44a3ae-4c83-4203-8b0b-116d6618accf-run-ovn\") pod \"ovnkube-node-cpwz4\" (UID: \"1c44a3ae-4c83-4203-8b0b-116d6618accf\") " pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.140682 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.140486 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1c44a3ae-4c83-4203-8b0b-116d6618accf-host-slash\") pod \"ovnkube-node-cpwz4\" (UID: \"1c44a3ae-4c83-4203-8b0b-116d6618accf\") " pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.140682 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.140501 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1c44a3ae-4c83-4203-8b0b-116d6618accf-host-cni-bin\") pod \"ovnkube-node-cpwz4\" (UID: \"1c44a3ae-4c83-4203-8b0b-116d6618accf\") " pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.140682 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.140534 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c44a3ae-4c83-4203-8b0b-116d6618accf-host-kubelet\") pod \"ovnkube-node-cpwz4\" (UID: \"1c44a3ae-4c83-4203-8b0b-116d6618accf\") " pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.140682 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.140547 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/da1371ec-f502-49c4-85e7-91e52e5d6ce2-var-lib-kubelet\") pod \"tuned-nglnq\" (UID: \"da1371ec-f502-49c4-85e7-91e52e5d6ce2\") " pod="openshift-cluster-node-tuning-operator/tuned-nglnq" Jul 17 07:11:37.141445 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.140591 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/19b25751-5376-4627-9965-d850af34a46d-cnibin\") pod \"multus-nfjfq\" (UID: \"19b25751-5376-4627-9965-d850af34a46d\") " pod="openshift-multus/multus-nfjfq" Jul 17 07:11:37.141445 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.140845 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1c44a3ae-4c83-4203-8b0b-116d6618accf-ovnkube-script-lib\") pod \"ovnkube-node-cpwz4\" (UID: \"1c44a3ae-4c83-4203-8b0b-116d6618accf\") " pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.141445 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.141023 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/19b25751-5376-4627-9965-d850af34a46d-multus-daemon-config\") pod \"multus-nfjfq\" (UID: \"19b25751-5376-4627-9965-d850af34a46d\") " pod="openshift-multus/multus-nfjfq" Jul 17 07:11:37.141445 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.141334 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1c44a3ae-4c83-4203-8b0b-116d6618accf-ovn-node-metrics-cert\") pod \"ovnkube-node-cpwz4\" (UID: \"1c44a3ae-4c83-4203-8b0b-116d6618accf\") " pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.142873 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:37.142853 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 17 07:11:37.142873 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:37.142876 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 17 07:11:37.143027 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:37.142889 2568 projected.go:196] Error preparing data for projected volume kube-api-access-9wmqr for pod openshift-network-diagnostics/network-check-target-dtjs7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 07:11:37.143027 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:37.142954 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376-kube-api-access-9wmqr podName:ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376 nodeName:}" failed. No retries permitted until 2026-07-17 07:11:37.642936095 +0000 UTC m=+2.127625062 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-9wmqr" (UniqueName: "kubernetes.io/projected/ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376-kube-api-access-9wmqr") pod "network-check-target-dtjs7" (UID: "ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 07:11:37.143337 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.143322 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/98ab979e-fe39-451d-b8d4-0098ea50c37f-agent-certs\") pod \"konnectivity-agent-zwtfj\" (UID: \"98ab979e-fe39-451d-b8d4-0098ea50c37f\") " pod="kube-system/konnectivity-agent-zwtfj" Jul 17 07:11:37.144999 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.144978 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6t6zk\" (UniqueName: \"kubernetes.io/projected/1c44a3ae-4c83-4203-8b0b-116d6618accf-kube-api-access-6t6zk\") pod \"ovnkube-node-cpwz4\" (UID: \"1c44a3ae-4c83-4203-8b0b-116d6618accf\") " pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.145612 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.145591 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbj7x\" (UniqueName: \"kubernetes.io/projected/3bc6540c-1f26-4a5e-9f66-e3480272699d-kube-api-access-qbj7x\") pod \"node-ca-cw57w\" (UID: \"3bc6540c-1f26-4a5e-9f66-e3480272699d\") " pod="openshift-image-registry/node-ca-cw57w" Jul 17 07:11:37.146937 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.146882 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkthr\" (UniqueName: \"kubernetes.io/projected/01aeee2e-a151-4e27-9ca7-8e837d05cb13-kube-api-access-tkthr\") pod \"iptables-alerter-774vt\" (UID: \"01aeee2e-a151-4e27-9ca7-8e837d05cb13\") " pod="openshift-network-operator/iptables-alerter-774vt" Jul 17 07:11:37.147736 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.147714 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jcw5\" (UniqueName: \"kubernetes.io/projected/19b25751-5376-4627-9965-d850af34a46d-kube-api-access-4jcw5\") pod \"multus-nfjfq\" (UID: \"19b25751-5376-4627-9965-d850af34a46d\") " pod="openshift-multus/multus-nfjfq" Jul 17 07:11:37.148904 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.148886 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wc2r7\" (UniqueName: \"kubernetes.io/projected/da1371ec-f502-49c4-85e7-91e52e5d6ce2-kube-api-access-wc2r7\") pod \"tuned-nglnq\" (UID: \"da1371ec-f502-49c4-85e7-91e52e5d6ce2\") " pod="openshift-cluster-node-tuning-operator/tuned-nglnq" Jul 17 07:11:37.151232 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.151217 2568 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 17 07:11:37.168529 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:37.168501 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6e4e077fdfb8959399f2ed04ffb3611.slice/crio-c96a9e44636fb90eb8d995eaf3946ec5f2123705c047fc268ae4e153ed672b4e WatchSource:0}: Error finding container c96a9e44636fb90eb8d995eaf3946ec5f2123705c047fc268ae4e153ed672b4e: Status 404 returned error can't find the container with id c96a9e44636fb90eb8d995eaf3946ec5f2123705c047fc268ae4e153ed672b4e Jul 17 07:11:37.170053 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:37.169905 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66a5663e408f43e5d633dcf4f079b8db.slice/crio-82ac66fedbffefc741ad5e8220e4f0e1da8b5b7e6ee8c27b2644181b44810026 WatchSource:0}: Error finding container 82ac66fedbffefc741ad5e8220e4f0e1da8b5b7e6ee8c27b2644181b44810026: Status 404 returned error can't find the container with id 82ac66fedbffefc741ad5e8220e4f0e1da8b5b7e6ee8c27b2644181b44810026 Jul 17 07:11:37.175094 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.175079 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 17 07:11:37.256268 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.256241 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qs6t2" Jul 17 07:11:37.262336 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.262315 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-t6g42" Jul 17 07:11:37.265700 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:37.265673 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c97ade1_4b91_446e_acca_12c828f10d6c.slice/crio-14ca90abd2eb210f8d418f072cdff77d546e650450cc642282c0e3eb8b3de295 WatchSource:0}: Error finding container 14ca90abd2eb210f8d418f072cdff77d546e650450cc642282c0e3eb8b3de295: Status 404 returned error can't find the container with id 14ca90abd2eb210f8d418f072cdff77d546e650450cc642282c0e3eb8b3de295 Jul 17 07:11:37.270439 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:37.270416 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a12d3cc_3a1a_4241_92b6_7d6198bc33c3.slice/crio-f9c0926dd090ae5474558d8ba937d46db1373580390c3963be292f93c0440d77 WatchSource:0}: Error finding container f9c0926dd090ae5474558d8ba937d46db1373580390c3963be292f93c0440d77: Status 404 returned error can't find the container with id f9c0926dd090ae5474558d8ba937d46db1373580390c3963be292f93c0440d77 Jul 17 07:11:37.298102 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.298084 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-z2q9x" Jul 17 07:11:37.306608 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:37.306587 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fbcebdd_a629_4d9a_9ba1_39e350bfa786.slice/crio-73e87bfad02517912aeee7f74eba49df1062fb2bb3affdb8e6a5839f2e7b2e6f WatchSource:0}: Error finding container 73e87bfad02517912aeee7f74eba49df1062fb2bb3affdb8e6a5839f2e7b2e6f: Status 404 returned error can't find the container with id 73e87bfad02517912aeee7f74eba49df1062fb2bb3affdb8e6a5839f2e7b2e6f Jul 17 07:11:37.316477 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.316458 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-774vt" Jul 17 07:11:37.323315 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.323299 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:11:37.325318 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:37.325301 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01aeee2e_a151_4e27_9ca7_8e837d05cb13.slice/crio-1fab59e3514bd2a1bf60f76a2f877932072c816d284f4a05feae831e69e96202 WatchSource:0}: Error finding container 1fab59e3514bd2a1bf60f76a2f877932072c816d284f4a05feae831e69e96202: Status 404 returned error can't find the container with id 1fab59e3514bd2a1bf60f76a2f877932072c816d284f4a05feae831e69e96202 Jul 17 07:11:37.330842 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:37.330822 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c44a3ae_4c83_4203_8b0b_116d6618accf.slice/crio-5143c2a4f94054f07473c9a7bb102a6a4a87943aaa1e852f0608e9b4ccdea8f2 WatchSource:0}: Error finding container 5143c2a4f94054f07473c9a7bb102a6a4a87943aaa1e852f0608e9b4ccdea8f2: Status 404 returned error can't find the container with id 5143c2a4f94054f07473c9a7bb102a6a4a87943aaa1e852f0608e9b4ccdea8f2 Jul 17 07:11:37.336660 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.336645 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zwtfj" Jul 17 07:11:37.342536 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.342515 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-nglnq" Jul 17 07:11:37.344295 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:37.344257 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98ab979e_fe39_451d_b8d4_0098ea50c37f.slice/crio-f33e8fdb8ae00c02d79903bb61a2421cacff626b03f088e5a29688225d3d7a1c WatchSource:0}: Error finding container f33e8fdb8ae00c02d79903bb61a2421cacff626b03f088e5a29688225d3d7a1c: Status 404 returned error can't find the container with id f33e8fdb8ae00c02d79903bb61a2421cacff626b03f088e5a29688225d3d7a1c Jul 17 07:11:37.350480 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:37.350462 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda1371ec_f502_49c4_85e7_91e52e5d6ce2.slice/crio-51c06a1886a287c27413ed6db63f3080d41b35b9fe31d40dda680398060fba41 WatchSource:0}: Error finding container 51c06a1886a287c27413ed6db63f3080d41b35b9fe31d40dda680398060fba41: Status 404 returned error can't find the container with id 51c06a1886a287c27413ed6db63f3080d41b35b9fe31d40dda680398060fba41 Jul 17 07:11:37.377784 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.377762 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-cw57w" Jul 17 07:11:37.380953 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.380938 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nfjfq" Jul 17 07:11:37.389324 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:37.389304 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bc6540c_1f26_4a5e_9f66_e3480272699d.slice/crio-e5caf7588d4338aa141ad357b99c84d4e923d1a9f4b0f865d4cd64dc5c76fe7f WatchSource:0}: Error finding container e5caf7588d4338aa141ad357b99c84d4e923d1a9f4b0f865d4cd64dc5c76fe7f: Status 404 returned error can't find the container with id e5caf7588d4338aa141ad357b99c84d4e923d1a9f4b0f865d4cd64dc5c76fe7f Jul 17 07:11:37.390652 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:11:37.390617 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19b25751_5376_4627_9965_d850af34a46d.slice/crio-bb7f73c2eba3e9e4d581e8534de9a9b290ad794ddfb6e366de208bfc7630623c WatchSource:0}: Error finding container bb7f73c2eba3e9e4d581e8534de9a9b290ad794ddfb6e366de208bfc7630623c: Status 404 returned error can't find the container with id bb7f73c2eba3e9e4d581e8534de9a9b290ad794ddfb6e366de208bfc7630623c Jul 17 07:11:37.543591 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.543506 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2639225a-fbee-4ebd-b36f-7ad47f365e79-original-pull-secret\") pod \"global-pull-secret-syncer-4x4z6\" (UID: \"2639225a-fbee-4ebd-b36f-7ad47f365e79\") " pod="kube-system/global-pull-secret-syncer-4x4z6" Jul 17 07:11:37.543591 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.543597 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/deb65c16-2d4c-40f0-aa9c-4399304b7222-metrics-certs\") pod \"network-metrics-daemon-rvx7b\" (UID: \"deb65c16-2d4c-40f0-aa9c-4399304b7222\") " pod="openshift-multus/network-metrics-daemon-rvx7b" Jul 17 07:11:37.543796 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:37.543660 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 17 07:11:37.543796 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:37.543673 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 07:11:37.543796 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:37.543720 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/deb65c16-2d4c-40f0-aa9c-4399304b7222-metrics-certs podName:deb65c16-2d4c-40f0-aa9c-4399304b7222 nodeName:}" failed. No retries permitted until 2026-07-17 07:11:38.543706199 +0000 UTC m=+3.028395165 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/deb65c16-2d4c-40f0-aa9c-4399304b7222-metrics-certs") pod "network-metrics-daemon-rvx7b" (UID: "deb65c16-2d4c-40f0-aa9c-4399304b7222") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 07:11:37.543796 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:37.543733 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2639225a-fbee-4ebd-b36f-7ad47f365e79-original-pull-secret podName:2639225a-fbee-4ebd-b36f-7ad47f365e79 nodeName:}" failed. No retries permitted until 2026-07-17 07:11:38.543727458 +0000 UTC m=+3.028416410 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2639225a-fbee-4ebd-b36f-7ad47f365e79-original-pull-secret") pod "global-pull-secret-syncer-4x4z6" (UID: "2639225a-fbee-4ebd-b36f-7ad47f365e79") : object "kube-system"/"original-pull-secret" not registered Jul 17 07:11:37.644425 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.644399 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wmqr\" (UniqueName: \"kubernetes.io/projected/ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376-kube-api-access-9wmqr\") pod \"network-check-target-dtjs7\" (UID: \"ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376\") " pod="openshift-network-diagnostics/network-check-target-dtjs7" Jul 17 07:11:37.644615 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:37.644538 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 17 07:11:37.644615 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:37.644557 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 17 07:11:37.644615 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:37.644569 2568 projected.go:196] Error preparing data for projected volume kube-api-access-9wmqr for pod openshift-network-diagnostics/network-check-target-dtjs7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 07:11:37.644780 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:37.644624 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376-kube-api-access-9wmqr podName:ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376 nodeName:}" failed. No retries permitted until 2026-07-17 07:11:38.644606616 +0000 UTC m=+3.129295567 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-9wmqr" (UniqueName: "kubernetes.io/projected/ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376-kube-api-access-9wmqr") pod "network-check-target-dtjs7" (UID: "ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 07:11:37.990364 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.978382 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-16 07:06:36 +0000 UTC" deadline="2027-12-21 18:14:14.550873643 +0000 UTC" Jul 17 07:11:37.990364 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.978440 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12539h2m36.572437904s" Jul 17 07:11:37.994415 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:37.993908 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rvx7b" Jul 17 07:11:37.994415 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:37.994044 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rvx7b" podUID="deb65c16-2d4c-40f0-aa9c-4399304b7222" Jul 17 07:11:38.011698 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:38.011641 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-233.ec2.internal" event={"ID":"c6e4e077fdfb8959399f2ed04ffb3611","Type":"ContainerStarted","Data":"c96a9e44636fb90eb8d995eaf3946ec5f2123705c047fc268ae4e153ed672b4e"} Jul 17 07:11:38.025742 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:38.025709 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-233.ec2.internal" event={"ID":"66a5663e408f43e5d633dcf4f079b8db","Type":"ContainerStarted","Data":"82ac66fedbffefc741ad5e8220e4f0e1da8b5b7e6ee8c27b2644181b44810026"} Jul 17 07:11:38.040815 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:38.040790 2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 17 07:11:38.044926 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:38.044894 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nfjfq" event={"ID":"19b25751-5376-4627-9965-d850af34a46d","Type":"ContainerStarted","Data":"bb7f73c2eba3e9e4d581e8534de9a9b290ad794ddfb6e366de208bfc7630623c"} Jul 17 07:11:38.057031 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:38.056998 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-cw57w" event={"ID":"3bc6540c-1f26-4a5e-9f66-e3480272699d","Type":"ContainerStarted","Data":"e5caf7588d4338aa141ad357b99c84d4e923d1a9f4b0f865d4cd64dc5c76fe7f"} Jul 17 07:11:38.070107 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:38.070073 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-nglnq" event={"ID":"da1371ec-f502-49c4-85e7-91e52e5d6ce2","Type":"ContainerStarted","Data":"51c06a1886a287c27413ed6db63f3080d41b35b9fe31d40dda680398060fba41"} Jul 17 07:11:38.084447 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:38.084417 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qs6t2" event={"ID":"8c97ade1-4b91-446e-acca-12c828f10d6c","Type":"ContainerStarted","Data":"14ca90abd2eb210f8d418f072cdff77d546e650450cc642282c0e3eb8b3de295"} Jul 17 07:11:38.094469 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:38.094397 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zwtfj" event={"ID":"98ab979e-fe39-451d-b8d4-0098ea50c37f","Type":"ContainerStarted","Data":"f33e8fdb8ae00c02d79903bb61a2421cacff626b03f088e5a29688225d3d7a1c"} Jul 17 07:11:38.106085 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:38.106058 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" event={"ID":"1c44a3ae-4c83-4203-8b0b-116d6618accf","Type":"ContainerStarted","Data":"5143c2a4f94054f07473c9a7bb102a6a4a87943aaa1e852f0608e9b4ccdea8f2"} Jul 17 07:11:38.123918 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:38.123831 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-774vt" event={"ID":"01aeee2e-a151-4e27-9ca7-8e837d05cb13","Type":"ContainerStarted","Data":"1fab59e3514bd2a1bf60f76a2f877932072c816d284f4a05feae831e69e96202"} Jul 17 07:11:38.132951 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:38.132921 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z2q9x" event={"ID":"4fbcebdd-a629-4d9a-9ba1-39e350bfa786","Type":"ContainerStarted","Data":"73e87bfad02517912aeee7f74eba49df1062fb2bb3affdb8e6a5839f2e7b2e6f"} Jul 17 07:11:38.148035 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:38.148008 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-t6g42" event={"ID":"6a12d3cc-3a1a-4241-92b6-7d6198bc33c3","Type":"ContainerStarted","Data":"f9c0926dd090ae5474558d8ba937d46db1373580390c3963be292f93c0440d77"} Jul 17 07:11:38.551041 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:38.551004 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/deb65c16-2d4c-40f0-aa9c-4399304b7222-metrics-certs\") pod \"network-metrics-daemon-rvx7b\" (UID: \"deb65c16-2d4c-40f0-aa9c-4399304b7222\") " pod="openshift-multus/network-metrics-daemon-rvx7b" Jul 17 07:11:38.551216 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:38.551053 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2639225a-fbee-4ebd-b36f-7ad47f365e79-original-pull-secret\") pod \"global-pull-secret-syncer-4x4z6\" (UID: \"2639225a-fbee-4ebd-b36f-7ad47f365e79\") " pod="kube-system/global-pull-secret-syncer-4x4z6" Jul 17 07:11:38.551216 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:38.551192 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 17 07:11:38.551374 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:38.551254 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2639225a-fbee-4ebd-b36f-7ad47f365e79-original-pull-secret podName:2639225a-fbee-4ebd-b36f-7ad47f365e79 nodeName:}" failed. No retries permitted until 2026-07-17 07:11:40.551235523 +0000 UTC m=+5.035924480 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2639225a-fbee-4ebd-b36f-7ad47f365e79-original-pull-secret") pod "global-pull-secret-syncer-4x4z6" (UID: "2639225a-fbee-4ebd-b36f-7ad47f365e79") : object "kube-system"/"original-pull-secret" not registered Jul 17 07:11:38.551680 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:38.551659 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 07:11:38.551764 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:38.551718 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/deb65c16-2d4c-40f0-aa9c-4399304b7222-metrics-certs podName:deb65c16-2d4c-40f0-aa9c-4399304b7222 nodeName:}" failed. No retries permitted until 2026-07-17 07:11:40.551702117 +0000 UTC m=+5.036391083 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/deb65c16-2d4c-40f0-aa9c-4399304b7222-metrics-certs") pod "network-metrics-daemon-rvx7b" (UID: "deb65c16-2d4c-40f0-aa9c-4399304b7222") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 07:11:38.651680 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:38.651646 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wmqr\" (UniqueName: \"kubernetes.io/projected/ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376-kube-api-access-9wmqr\") pod \"network-check-target-dtjs7\" (UID: \"ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376\") " pod="openshift-network-diagnostics/network-check-target-dtjs7" Jul 17 07:11:38.651830 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:38.651797 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 17 07:11:38.651830 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:38.651816 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 17 07:11:38.651830 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:38.651827 2568 projected.go:196] Error preparing data for projected volume kube-api-access-9wmqr for pod openshift-network-diagnostics/network-check-target-dtjs7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 07:11:38.651994 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:38.651898 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376-kube-api-access-9wmqr podName:ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376 nodeName:}" failed. No retries permitted until 2026-07-17 07:11:40.651878352 +0000 UTC m=+5.136567323 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-9wmqr" (UniqueName: "kubernetes.io/projected/ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376-kube-api-access-9wmqr") pod "network-check-target-dtjs7" (UID: "ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 07:11:38.979652 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:38.979534 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-16 07:06:36 +0000 UTC" deadline="2028-03-10 16:37:59.355950806 +0000 UTC" Jul 17 07:11:38.979652 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:38.979599 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14457h26m20.376356092s" Jul 17 07:11:38.994020 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:38.993990 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4x4z6" Jul 17 07:11:38.994456 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:38.994125 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4x4z6" podUID="2639225a-fbee-4ebd-b36f-7ad47f365e79" Jul 17 07:11:38.994584 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:38.994565 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dtjs7" Jul 17 07:11:38.994688 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:38.994662 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dtjs7" podUID="ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376" Jul 17 07:11:39.270137 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:39.270054 2568 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 17 07:11:39.585563 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:39.585480 2568 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 17 07:11:39.995945 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:39.995869 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rvx7b" Jul 17 07:11:39.996367 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:39.996015 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rvx7b" podUID="deb65c16-2d4c-40f0-aa9c-4399304b7222" Jul 17 07:11:40.566540 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:40.566511 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/deb65c16-2d4c-40f0-aa9c-4399304b7222-metrics-certs\") pod \"network-metrics-daemon-rvx7b\" (UID: \"deb65c16-2d4c-40f0-aa9c-4399304b7222\") " pod="openshift-multus/network-metrics-daemon-rvx7b" Jul 17 07:11:40.566540 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:40.566550 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2639225a-fbee-4ebd-b36f-7ad47f365e79-original-pull-secret\") pod \"global-pull-secret-syncer-4x4z6\" (UID: \"2639225a-fbee-4ebd-b36f-7ad47f365e79\") " pod="kube-system/global-pull-secret-syncer-4x4z6" Jul 17 07:11:40.566799 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:40.566682 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 07:11:40.566799 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:40.566704 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 17 07:11:40.566799 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:40.566753 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/deb65c16-2d4c-40f0-aa9c-4399304b7222-metrics-certs podName:deb65c16-2d4c-40f0-aa9c-4399304b7222 nodeName:}" failed. No retries permitted until 2026-07-17 07:11:44.566734055 +0000 UTC m=+9.051423010 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/deb65c16-2d4c-40f0-aa9c-4399304b7222-metrics-certs") pod "network-metrics-daemon-rvx7b" (UID: "deb65c16-2d4c-40f0-aa9c-4399304b7222") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 07:11:40.566799 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:40.566773 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2639225a-fbee-4ebd-b36f-7ad47f365e79-original-pull-secret podName:2639225a-fbee-4ebd-b36f-7ad47f365e79 nodeName:}" failed. No retries permitted until 2026-07-17 07:11:44.566762905 +0000 UTC m=+9.051451862 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2639225a-fbee-4ebd-b36f-7ad47f365e79-original-pull-secret") pod "global-pull-secret-syncer-4x4z6" (UID: "2639225a-fbee-4ebd-b36f-7ad47f365e79") : object "kube-system"/"original-pull-secret" not registered Jul 17 07:11:40.667749 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:40.667715 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wmqr\" (UniqueName: \"kubernetes.io/projected/ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376-kube-api-access-9wmqr\") pod \"network-check-target-dtjs7\" (UID: \"ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376\") " pod="openshift-network-diagnostics/network-check-target-dtjs7" Jul 17 07:11:40.667912 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:40.667893 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 17 07:11:40.667912 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:40.667910 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 17 07:11:40.668033 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:40.667923 2568 projected.go:196] Error preparing data for projected volume kube-api-access-9wmqr for pod openshift-network-diagnostics/network-check-target-dtjs7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 07:11:40.668033 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:40.667978 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376-kube-api-access-9wmqr podName:ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376 nodeName:}" failed. No retries permitted until 2026-07-17 07:11:44.667958972 +0000 UTC m=+9.152647936 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-9wmqr" (UniqueName: "kubernetes.io/projected/ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376-kube-api-access-9wmqr") pod "network-check-target-dtjs7" (UID: "ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 07:11:40.995357 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:40.995324 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4x4z6" Jul 17 07:11:40.995506 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:40.995458 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4x4z6" podUID="2639225a-fbee-4ebd-b36f-7ad47f365e79" Jul 17 07:11:40.996091 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:40.996058 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dtjs7" Jul 17 07:11:40.996599 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:40.996164 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dtjs7" podUID="ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376" Jul 17 07:11:41.994627 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:41.994595 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rvx7b" Jul 17 07:11:41.994817 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:41.994744 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rvx7b" podUID="deb65c16-2d4c-40f0-aa9c-4399304b7222" Jul 17 07:11:42.993231 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:42.993196 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dtjs7" Jul 17 07:11:42.993729 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:42.993346 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dtjs7" podUID="ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376" Jul 17 07:11:42.993729 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:42.993416 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4x4z6" Jul 17 07:11:42.993729 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:42.993513 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4x4z6" podUID="2639225a-fbee-4ebd-b36f-7ad47f365e79" Jul 17 07:11:43.993735 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:43.993705 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rvx7b" Jul 17 07:11:43.994168 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:43.993849 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rvx7b" podUID="deb65c16-2d4c-40f0-aa9c-4399304b7222" Jul 17 07:11:44.596973 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:44.596833 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/deb65c16-2d4c-40f0-aa9c-4399304b7222-metrics-certs\") pod \"network-metrics-daemon-rvx7b\" (UID: \"deb65c16-2d4c-40f0-aa9c-4399304b7222\") " pod="openshift-multus/network-metrics-daemon-rvx7b" Jul 17 07:11:44.596973 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:44.596885 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2639225a-fbee-4ebd-b36f-7ad47f365e79-original-pull-secret\") pod \"global-pull-secret-syncer-4x4z6\" (UID: \"2639225a-fbee-4ebd-b36f-7ad47f365e79\") " pod="kube-system/global-pull-secret-syncer-4x4z6" Jul 17 07:11:44.597206 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:44.597004 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 07:11:44.597206 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:44.597036 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 17 07:11:44.597206 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:44.597079 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/deb65c16-2d4c-40f0-aa9c-4399304b7222-metrics-certs podName:deb65c16-2d4c-40f0-aa9c-4399304b7222 nodeName:}" failed. No retries permitted until 2026-07-17 07:11:52.59705849 +0000 UTC m=+17.081747456 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/deb65c16-2d4c-40f0-aa9c-4399304b7222-metrics-certs") pod "network-metrics-daemon-rvx7b" (UID: "deb65c16-2d4c-40f0-aa9c-4399304b7222") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 07:11:44.597206 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:44.597100 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2639225a-fbee-4ebd-b36f-7ad47f365e79-original-pull-secret podName:2639225a-fbee-4ebd-b36f-7ad47f365e79 nodeName:}" failed. No retries permitted until 2026-07-17 07:11:52.597089472 +0000 UTC m=+17.081778430 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2639225a-fbee-4ebd-b36f-7ad47f365e79-original-pull-secret") pod "global-pull-secret-syncer-4x4z6" (UID: "2639225a-fbee-4ebd-b36f-7ad47f365e79") : object "kube-system"/"original-pull-secret" not registered Jul 17 07:11:44.697932 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:44.697833 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wmqr\" (UniqueName: \"kubernetes.io/projected/ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376-kube-api-access-9wmqr\") pod \"network-check-target-dtjs7\" (UID: \"ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376\") " pod="openshift-network-diagnostics/network-check-target-dtjs7" Jul 17 07:11:44.698088 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:44.698073 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 17 07:11:44.698161 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:44.698096 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 17 07:11:44.698161 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:44.698109 2568 projected.go:196] Error preparing data for projected volume kube-api-access-9wmqr for pod openshift-network-diagnostics/network-check-target-dtjs7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 07:11:44.698251 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:44.698177 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376-kube-api-access-9wmqr podName:ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376 nodeName:}" failed. No retries permitted until 2026-07-17 07:11:52.698157019 +0000 UTC m=+17.182845984 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-9wmqr" (UniqueName: "kubernetes.io/projected/ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376-kube-api-access-9wmqr") pod "network-check-target-dtjs7" (UID: "ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 07:11:44.994329 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:44.994138 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4x4z6" Jul 17 07:11:44.994329 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:44.994272 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4x4z6" podUID="2639225a-fbee-4ebd-b36f-7ad47f365e79" Jul 17 07:11:44.994810 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:44.994340 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dtjs7" Jul 17 07:11:44.994810 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:44.994457 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dtjs7" podUID="ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376" Jul 17 07:11:45.994569 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:45.994535 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rvx7b" Jul 17 07:11:45.995070 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:45.994668 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rvx7b" podUID="deb65c16-2d4c-40f0-aa9c-4399304b7222" Jul 17 07:11:46.994266 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:46.994231 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4x4z6" Jul 17 07:11:46.994463 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:46.994279 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dtjs7" Jul 17 07:11:46.994463 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:46.994395 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4x4z6" podUID="2639225a-fbee-4ebd-b36f-7ad47f365e79" Jul 17 07:11:46.994597 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:46.994524 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dtjs7" podUID="ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376" Jul 17 07:11:47.994178 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:47.994144 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rvx7b" Jul 17 07:11:47.994386 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:47.994321 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rvx7b" podUID="deb65c16-2d4c-40f0-aa9c-4399304b7222" Jul 17 07:11:48.993497 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:48.993441 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4x4z6" Jul 17 07:11:48.993892 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:48.993441 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dtjs7" Jul 17 07:11:48.993892 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:48.993556 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4x4z6" podUID="2639225a-fbee-4ebd-b36f-7ad47f365e79" Jul 17 07:11:48.993892 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:48.993668 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dtjs7" podUID="ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376" Jul 17 07:11:49.993764 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:49.993720 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rvx7b" Jul 17 07:11:49.994201 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:49.993851 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rvx7b" podUID="deb65c16-2d4c-40f0-aa9c-4399304b7222" Jul 17 07:11:50.993791 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:50.993749 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4x4z6" Jul 17 07:11:50.994211 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:50.993866 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4x4z6" podUID="2639225a-fbee-4ebd-b36f-7ad47f365e79" Jul 17 07:11:50.994211 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:50.993929 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dtjs7" Jul 17 07:11:50.994211 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:50.994038 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dtjs7" podUID="ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376" Jul 17 07:11:51.993730 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:51.993687 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rvx7b" Jul 17 07:11:51.993924 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:51.993834 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rvx7b" podUID="deb65c16-2d4c-40f0-aa9c-4399304b7222" Jul 17 07:11:52.659355 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:52.659322 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/deb65c16-2d4c-40f0-aa9c-4399304b7222-metrics-certs\") pod \"network-metrics-daemon-rvx7b\" (UID: \"deb65c16-2d4c-40f0-aa9c-4399304b7222\") " pod="openshift-multus/network-metrics-daemon-rvx7b" Jul 17 07:11:52.659355 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:52.659355 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2639225a-fbee-4ebd-b36f-7ad47f365e79-original-pull-secret\") pod \"global-pull-secret-syncer-4x4z6\" (UID: \"2639225a-fbee-4ebd-b36f-7ad47f365e79\") " pod="kube-system/global-pull-secret-syncer-4x4z6" Jul 17 07:11:52.659554 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:52.659479 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 07:11:52.659554 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:52.659525 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 17 07:11:52.659554 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:52.659543 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/deb65c16-2d4c-40f0-aa9c-4399304b7222-metrics-certs podName:deb65c16-2d4c-40f0-aa9c-4399304b7222 nodeName:}" failed. No retries permitted until 2026-07-17 07:12:08.659527315 +0000 UTC m=+33.144216271 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/deb65c16-2d4c-40f0-aa9c-4399304b7222-metrics-certs") pod "network-metrics-daemon-rvx7b" (UID: "deb65c16-2d4c-40f0-aa9c-4399304b7222") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 07:11:52.659664 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:52.659573 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2639225a-fbee-4ebd-b36f-7ad47f365e79-original-pull-secret podName:2639225a-fbee-4ebd-b36f-7ad47f365e79 nodeName:}" failed. No retries permitted until 2026-07-17 07:12:08.659559784 +0000 UTC m=+33.144248739 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2639225a-fbee-4ebd-b36f-7ad47f365e79-original-pull-secret") pod "global-pull-secret-syncer-4x4z6" (UID: "2639225a-fbee-4ebd-b36f-7ad47f365e79") : object "kube-system"/"original-pull-secret" not registered Jul 17 07:11:52.759920 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:52.759888 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wmqr\" (UniqueName: \"kubernetes.io/projected/ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376-kube-api-access-9wmqr\") pod \"network-check-target-dtjs7\" (UID: \"ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376\") " pod="openshift-network-diagnostics/network-check-target-dtjs7" Jul 17 07:11:52.760097 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:52.760022 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 17 07:11:52.760097 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:52.760037 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 17 07:11:52.760097 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:52.760045 2568 projected.go:196] Error preparing data for projected volume kube-api-access-9wmqr for pod openshift-network-diagnostics/network-check-target-dtjs7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 07:11:52.760097 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:52.760095 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376-kube-api-access-9wmqr podName:ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376 nodeName:}" failed. No retries permitted until 2026-07-17 07:12:08.760080351 +0000 UTC m=+33.244769321 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-9wmqr" (UniqueName: "kubernetes.io/projected/ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376-kube-api-access-9wmqr") pod "network-check-target-dtjs7" (UID: "ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 07:11:52.993631 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:52.993563 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dtjs7" Jul 17 07:11:52.993631 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:52.993593 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4x4z6" Jul 17 07:11:52.993807 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:52.993670 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dtjs7" podUID="ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376" Jul 17 07:11:52.993807 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:52.993786 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4x4z6" podUID="2639225a-fbee-4ebd-b36f-7ad47f365e79" Jul 17 07:11:53.993278 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:53.993245 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rvx7b" Jul 17 07:11:53.993762 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:53.993413 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rvx7b" podUID="deb65c16-2d4c-40f0-aa9c-4399304b7222" Jul 17 07:11:54.993914 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:54.993881 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dtjs7" Jul 17 07:11:54.994406 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:54.993884 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4x4z6" Jul 17 07:11:54.994406 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:54.994007 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dtjs7" podUID="ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376" Jul 17 07:11:54.994406 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:54.994093 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4x4z6" podUID="2639225a-fbee-4ebd-b36f-7ad47f365e79" Jul 17 07:11:55.994876 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:55.994612 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rvx7b" Jul 17 07:11:55.995379 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:55.994997 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rvx7b" podUID="deb65c16-2d4c-40f0-aa9c-4399304b7222" Jul 17 07:11:56.993310 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:56.993262 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dtjs7" Jul 17 07:11:56.993480 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:56.993267 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4x4z6" Jul 17 07:11:56.993480 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:56.993380 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dtjs7" podUID="ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376" Jul 17 07:11:56.993566 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:56.993475 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4x4z6" podUID="2639225a-fbee-4ebd-b36f-7ad47f365e79" Jul 17 07:11:57.993528 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:57.993503 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rvx7b" Jul 17 07:11:57.993978 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:57.993642 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rvx7b" podUID="deb65c16-2d4c-40f0-aa9c-4399304b7222" Jul 17 07:11:58.196925 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:58.196766 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-233.ec2.internal" event={"ID":"66a5663e408f43e5d633dcf4f079b8db","Type":"ContainerStarted","Data":"2bbdbba6f9db13672f93df1594517f165642883b32ccb842c58c66753fad499b"} Jul 17 07:11:58.198545 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:58.198514 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nfjfq" event={"ID":"19b25751-5376-4627-9965-d850af34a46d","Type":"ContainerStarted","Data":"ae888dac17580a297ec318d49af76c230fe4c9cebe00e89388bfcdbf877006ba"} Jul 17 07:11:58.200059 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:58.200036 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-nglnq" event={"ID":"da1371ec-f502-49c4-85e7-91e52e5d6ce2","Type":"ContainerStarted","Data":"8570efdc5979b0ed428055c9aaf044ef0638d05c015d54c34e25443fe9366e5d"} Jul 17 07:11:58.201814 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:58.201708 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cpwz4_1c44a3ae-4c83-4203-8b0b-116d6618accf/ovn-acl-logging/0.log" Jul 17 07:11:58.202171 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:58.202070 2568 generic.go:356] "Generic (PLEG): container finished" podID="1c44a3ae-4c83-4203-8b0b-116d6618accf" containerID="9f4ec1cda7ca05d16df610995811f5aed9576699dd7eac4a6777019182592b8d" exitCode=1 Jul 17 07:11:58.202171 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:58.202108 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" event={"ID":"1c44a3ae-4c83-4203-8b0b-116d6618accf","Type":"ContainerDied","Data":"9f4ec1cda7ca05d16df610995811f5aed9576699dd7eac4a6777019182592b8d"} Jul 17 07:11:58.202306 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:58.202183 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" event={"ID":"1c44a3ae-4c83-4203-8b0b-116d6618accf","Type":"ContainerStarted","Data":"9600121b95d51e684325078af62ba86ff685e2a96b048a5994cafe041be3ab30"} Jul 17 07:11:58.218788 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:58.218732 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-233.ec2.internal" podStartSLOduration=22.218715175 podStartE2EDuration="22.218715175s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 07:11:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:11:58.218710907 +0000 UTC m=+22.703399882" watchObservedRunningTime="2026-07-17 07:11:58.218715175 +0000 UTC m=+22.703404147" Jul 17 07:11:58.241298 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:58.241231 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-nglnq" podStartSLOduration=2.364417158 podStartE2EDuration="22.241210365s" totalImagesPullingTime="19.876793207s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 07:11:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:11:58.241017577 +0000 UTC m=+22.725706551" watchObservedRunningTime="2026-07-17 07:11:58.241210365 +0000 UTC m=+22.725899344" Jul 17 07:11:58.259936 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:58.259896 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-nfjfq" podStartSLOduration=2.393254308 podStartE2EDuration="22.25988142s" totalImagesPullingTime="19.866627112s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 07:11:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:11:58.259311405 +0000 UTC m=+22.744000369" watchObservedRunningTime="2026-07-17 07:11:58.25988142 +0000 UTC m=+22.744570390" Jul 17 07:11:58.993503 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:58.993467 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4x4z6" Jul 17 07:11:58.993705 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:58.993589 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4x4z6" podUID="2639225a-fbee-4ebd-b36f-7ad47f365e79" Jul 17 07:11:58.993705 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:58.993632 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dtjs7" Jul 17 07:11:58.994373 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:58.993737 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dtjs7" podUID="ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376" Jul 17 07:11:59.206396 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:59.206140 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-cw57w" event={"ID":"3bc6540c-1f26-4a5e-9f66-e3480272699d","Type":"ContainerStarted","Data":"24c597d309378a5732b6aa0de843850699353dde09ddb04ba1ac957de62b437d"} Jul 17 07:11:59.207878 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:59.207830 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qs6t2" event={"ID":"8c97ade1-4b91-446e-acca-12c828f10d6c","Type":"ContainerStarted","Data":"240c2b25fee1fd0f37eb8329ecd363df93449b415058d139e6158e24c83d1865"} Jul 17 07:11:59.210622 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:59.209819 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zwtfj" event={"ID":"98ab979e-fe39-451d-b8d4-0098ea50c37f","Type":"ContainerStarted","Data":"6b711389d24a9e89ea23a5fceeb7d1cce45dd5d2333fc17ec7642fd1a24b66ec"} Jul 17 07:11:59.213907 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:59.213885 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cpwz4_1c44a3ae-4c83-4203-8b0b-116d6618accf/ovn-acl-logging/0.log" Jul 17 07:11:59.214388 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:59.214362 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" event={"ID":"1c44a3ae-4c83-4203-8b0b-116d6618accf","Type":"ContainerStarted","Data":"d74cb0da757489fe178ec906da2b3c0c97ce2d5f6491b4ad4bc4993ffb293f2c"} Jul 17 07:11:59.214489 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:59.214451 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" event={"ID":"1c44a3ae-4c83-4203-8b0b-116d6618accf","Type":"ContainerStarted","Data":"c5edbdbdaa60f7cf9ff0acee0874f604f9c6e3fc063de7bea4221b889ef0f9ff"} Jul 17 07:11:59.214539 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:59.214503 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" event={"ID":"1c44a3ae-4c83-4203-8b0b-116d6618accf","Type":"ContainerStarted","Data":"a5e61cd39819ca12e5e5cff551613bd379f8e8bd9509bc86b5668c622fea973e"} Jul 17 07:11:59.214590 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:59.214554 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" event={"ID":"1c44a3ae-4c83-4203-8b0b-116d6618accf","Type":"ContainerStarted","Data":"a931ea336588f3e628d5d7e7477a4a13fff812ac3de37d79cb04ed504f9281aa"} Jul 17 07:11:59.216049 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:59.216018 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-774vt" event={"ID":"01aeee2e-a151-4e27-9ca7-8e837d05cb13","Type":"ContainerStarted","Data":"9814475a84718c64ef17b8fdfe74cd504a896352d438c5a6550a18e8019a451a"} Jul 17 07:11:59.218005 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:59.217981 2568 generic.go:356] "Generic (PLEG): container finished" podID="4fbcebdd-a629-4d9a-9ba1-39e350bfa786" containerID="7e2d0b9029b8fa6fb06a77f349250c3a77e2049d4c51a894d2ecf48a1a238240" exitCode=0 Jul 17 07:11:59.218108 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:59.218059 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z2q9x" event={"ID":"4fbcebdd-a629-4d9a-9ba1-39e350bfa786","Type":"ContainerDied","Data":"7e2d0b9029b8fa6fb06a77f349250c3a77e2049d4c51a894d2ecf48a1a238240"} Jul 17 07:11:59.219619 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:59.219601 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-t6g42" event={"ID":"6a12d3cc-3a1a-4241-92b6-7d6198bc33c3","Type":"ContainerStarted","Data":"f14c0929bb71043306cb044624044cea140d62e703d7d51c5302f1d891cd6cb6"} Jul 17 07:11:59.221242 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:59.221218 2568 generic.go:356] "Generic (PLEG): container finished" podID="c6e4e077fdfb8959399f2ed04ffb3611" containerID="6c57033e5c46091946125e8a676bfd38eda70d5ab744b2558a2466180806095c" exitCode=0 Jul 17 07:11:59.221416 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:59.221390 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-233.ec2.internal" event={"ID":"c6e4e077fdfb8959399f2ed04ffb3611","Type":"ContainerDied","Data":"6c57033e5c46091946125e8a676bfd38eda70d5ab744b2558a2466180806095c"} Jul 17 07:11:59.242662 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:59.242612 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-cw57w" podStartSLOduration=3.406105786 podStartE2EDuration="23.24258836s" totalImagesPullingTime="19.836482574s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 07:11:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:11:59.224764794 +0000 UTC m=+23.709453769" watchObservedRunningTime="2026-07-17 07:11:59.24258836 +0000 UTC m=+23.727277347" Jul 17 07:11:59.262907 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:59.262803 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-zwtfj" podStartSLOduration=3.395774679 podStartE2EDuration="23.262792923s" totalImagesPullingTime="19.867018244s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 07:11:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:11:59.243124823 +0000 UTC m=+23.727813798" watchObservedRunningTime="2026-07-17 07:11:59.262792923 +0000 UTC m=+23.747481901" Jul 17 07:11:59.263176 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:59.263156 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-t6g42" podStartSLOduration=3.308152039 podStartE2EDuration="23.263151022s" totalImagesPullingTime="19.954998983s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 07:11:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:11:59.262711939 +0000 UTC m=+23.747400914" watchObservedRunningTime="2026-07-17 07:11:59.263151022 +0000 UTC m=+23.747839995" Jul 17 07:11:59.308368 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:59.308328 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-774vt" podStartSLOduration=3.408211091 podStartE2EDuration="23.308312701s" totalImagesPullingTime="19.90010161s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 07:11:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:11:59.308307615 +0000 UTC m=+23.792996588" watchObservedRunningTime="2026-07-17 07:11:59.308312701 +0000 UTC m=+23.793001668" Jul 17 07:11:59.579462 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:59.579435 2568 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 17 07:11:59.993426 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:11:59.993354 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rvx7b" Jul 17 07:11:59.993687 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:11:59.993479 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rvx7b" podUID="deb65c16-2d4c-40f0-aa9c-4399304b7222" Jul 17 07:12:00.095594 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:00.095485 2568 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-17T07:11:59.579457642Z","UUID":"a8701269-cc12-4565-89f0-51fccded9e23","Handler":null,"Name":"","Endpoint":""} Jul 17 07:12:00.097114 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:00.097090 2568 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 17 07:12:00.097240 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:00.097131 2568 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 17 07:12:00.226237 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:00.226191 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-233.ec2.internal" event={"ID":"c6e4e077fdfb8959399f2ed04ffb3611","Type":"ContainerStarted","Data":"8d12f7216cf6779ed0c3b39cf20693aa4e654a76a3c6f22a43db8e2ce9fb58fc"} Jul 17 07:12:00.228941 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:00.228873 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qs6t2" event={"ID":"8c97ade1-4b91-446e-acca-12c828f10d6c","Type":"ContainerStarted","Data":"d0d9d5ec78f8fc5ff073b0b818615d840cb82fa8f92224f1d3cf5d4b55694b4e"} Jul 17 07:12:00.994089 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:00.994017 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4x4z6" Jul 17 07:12:00.994357 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:00.994017 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dtjs7" Jul 17 07:12:00.994357 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:00.994132 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4x4z6" podUID="2639225a-fbee-4ebd-b36f-7ad47f365e79" Jul 17 07:12:00.994357 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:00.994239 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dtjs7" podUID="ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376" Jul 17 07:12:01.233211 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:01.233170 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qs6t2" event={"ID":"8c97ade1-4b91-446e-acca-12c828f10d6c","Type":"ContainerStarted","Data":"5bfad927aafdd960428d1222d8da75f6009f06baa750cfe47b93b026beaefee1"} Jul 17 07:12:01.236545 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:01.236525 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cpwz4_1c44a3ae-4c83-4203-8b0b-116d6618accf/ovn-acl-logging/0.log" Jul 17 07:12:01.236841 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:01.236818 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" event={"ID":"1c44a3ae-4c83-4203-8b0b-116d6618accf","Type":"ContainerStarted","Data":"89c69a889e43b2ad594a77d85b7242e85a874db690e441c6ab64bbb583cd1928"} Jul 17 07:12:01.251572 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:01.251490 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-233.ec2.internal" podStartSLOduration=25.251477129 podStartE2EDuration="25.251477129s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 07:11:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:12:00.241467729 +0000 UTC m=+24.726156705" watchObservedRunningTime="2026-07-17 07:12:01.251477129 +0000 UTC m=+25.736166103" Jul 17 07:12:01.251778 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:01.251751 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qs6t2" podStartSLOduration=2.952670187 podStartE2EDuration="25.251742807s" totalImagesPullingTime="22.29907262s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 07:11:36 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:12:01.251604115 +0000 UTC m=+25.736293067" watchObservedRunningTime="2026-07-17 07:12:01.251742807 +0000 UTC m=+25.736431788" Jul 17 07:12:01.542517 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:01.542440 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-zwtfj" Jul 17 07:12:01.994044 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:01.993966 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rvx7b" Jul 17 07:12:01.994203 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:01.994135 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rvx7b" podUID="deb65c16-2d4c-40f0-aa9c-4399304b7222" Jul 17 07:12:02.993567 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:02.993524 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4x4z6" Jul 17 07:12:02.993567 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:02.993567 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dtjs7" Jul 17 07:12:02.994101 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:02.993680 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4x4z6" podUID="2639225a-fbee-4ebd-b36f-7ad47f365e79" Jul 17 07:12:02.994101 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:02.993801 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dtjs7" podUID="ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376" Jul 17 07:12:03.564253 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:03.564222 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-zwtfj" Jul 17 07:12:03.564897 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:03.564877 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-zwtfj" Jul 17 07:12:03.993421 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:03.993343 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rvx7b" Jul 17 07:12:03.993574 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:03.993467 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rvx7b" podUID="deb65c16-2d4c-40f0-aa9c-4399304b7222" Jul 17 07:12:04.243619 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:04.243551 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-zwtfj" Jul 17 07:12:04.994112 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:04.993876 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4x4z6" Jul 17 07:12:04.994306 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:04.993933 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dtjs7" Jul 17 07:12:04.994306 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:04.994153 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4x4z6" podUID="2639225a-fbee-4ebd-b36f-7ad47f365e79" Jul 17 07:12:04.994306 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:04.994199 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dtjs7" podUID="ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376" Jul 17 07:12:05.248520 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:05.248445 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cpwz4_1c44a3ae-4c83-4203-8b0b-116d6618accf/ovn-acl-logging/0.log" Jul 17 07:12:05.249176 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:05.248792 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" event={"ID":"1c44a3ae-4c83-4203-8b0b-116d6618accf","Type":"ContainerStarted","Data":"cf7422aaa52ae88ab7beec7b0fc60274c8e884ae4c561736d46a9b68ae29e3af"} Jul 17 07:12:05.249176 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:05.249052 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:12:05.249338 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:05.249322 2568 scope.go:117] "RemoveContainer" containerID="9f4ec1cda7ca05d16df610995811f5aed9576699dd7eac4a6777019182592b8d" Jul 17 07:12:05.251225 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:05.251199 2568 generic.go:356] "Generic (PLEG): container finished" podID="4fbcebdd-a629-4d9a-9ba1-39e350bfa786" containerID="f370c3204f748bafecfa14ca0f402d2a3e051e38dceafc15b6def2c60fffa5c5" exitCode=0 Jul 17 07:12:05.252887 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:05.252750 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z2q9x" event={"ID":"4fbcebdd-a629-4d9a-9ba1-39e350bfa786","Type":"ContainerDied","Data":"f370c3204f748bafecfa14ca0f402d2a3e051e38dceafc15b6def2c60fffa5c5"} Jul 17 07:12:05.266229 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:05.266209 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:12:05.994846 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:05.994814 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rvx7b" Jul 17 07:12:05.995269 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:05.994941 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rvx7b" podUID="deb65c16-2d4c-40f0-aa9c-4399304b7222" Jul 17 07:12:06.256895 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:06.256797 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cpwz4_1c44a3ae-4c83-4203-8b0b-116d6618accf/ovn-acl-logging/0.log" Jul 17 07:12:06.257253 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:06.257215 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" event={"ID":"1c44a3ae-4c83-4203-8b0b-116d6618accf","Type":"ContainerStarted","Data":"57990fb1b9e059d36f1d792a7f6be008ebec03442a07335181c8fc7cd01ef5c9"} Jul 17 07:12:06.257518 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:06.257497 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:12:06.257656 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:06.257587 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:12:06.273079 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:06.273055 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:12:06.285894 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:06.285853 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" podStartSLOduration=9.737793357 podStartE2EDuration="30.285839523s" totalImagesPullingTime="20.548046166s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 07:11:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:12:06.28446063 +0000 UTC m=+30.769149605" watchObservedRunningTime="2026-07-17 07:12:06.285839523 +0000 UTC m=+30.770528497" Jul 17 07:12:06.609832 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:06.609759 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rvx7b"] Jul 17 07:12:06.609996 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:06.609900 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rvx7b" Jul 17 07:12:06.610049 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:06.610002 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rvx7b" podUID="deb65c16-2d4c-40f0-aa9c-4399304b7222" Jul 17 07:12:06.612581 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:06.612560 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4x4z6"] Jul 17 07:12:06.612669 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:06.612648 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4x4z6" Jul 17 07:12:06.612741 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:06.612724 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4x4z6" podUID="2639225a-fbee-4ebd-b36f-7ad47f365e79" Jul 17 07:12:06.615544 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:06.615516 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-dtjs7"] Jul 17 07:12:06.615650 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:06.615610 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dtjs7" Jul 17 07:12:06.615699 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:06.615684 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dtjs7" podUID="ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376" Jul 17 07:12:07.261695 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:07.261600 2568 generic.go:356] "Generic (PLEG): container finished" podID="4fbcebdd-a629-4d9a-9ba1-39e350bfa786" containerID="db49d78d7261e12ef93ce959532d5620e3477d9c0daf8a4ef8516b9c1780ed65" exitCode=0 Jul 17 07:12:07.261695 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:07.261682 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z2q9x" event={"ID":"4fbcebdd-a629-4d9a-9ba1-39e350bfa786","Type":"ContainerDied","Data":"db49d78d7261e12ef93ce959532d5620e3477d9c0daf8a4ef8516b9c1780ed65"} Jul 17 07:12:07.994016 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:07.993937 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dtjs7" Jul 17 07:12:07.994172 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:07.994089 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dtjs7" podUID="ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376" Jul 17 07:12:07.994172 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:07.994128 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4x4z6" Jul 17 07:12:07.994262 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:07.994241 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4x4z6" podUID="2639225a-fbee-4ebd-b36f-7ad47f365e79" Jul 17 07:12:08.675762 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:08.675682 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/deb65c16-2d4c-40f0-aa9c-4399304b7222-metrics-certs\") pod \"network-metrics-daemon-rvx7b\" (UID: \"deb65c16-2d4c-40f0-aa9c-4399304b7222\") " pod="openshift-multus/network-metrics-daemon-rvx7b" Jul 17 07:12:08.675762 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:08.675717 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2639225a-fbee-4ebd-b36f-7ad47f365e79-original-pull-secret\") pod \"global-pull-secret-syncer-4x4z6\" (UID: \"2639225a-fbee-4ebd-b36f-7ad47f365e79\") " pod="kube-system/global-pull-secret-syncer-4x4z6" Jul 17 07:12:08.676120 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:08.675827 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 17 07:12:08.676120 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:08.675831 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 07:12:08.676120 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:08.675874 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2639225a-fbee-4ebd-b36f-7ad47f365e79-original-pull-secret podName:2639225a-fbee-4ebd-b36f-7ad47f365e79 nodeName:}" failed. No retries permitted until 2026-07-17 07:12:40.675862463 +0000 UTC m=+65.160551415 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2639225a-fbee-4ebd-b36f-7ad47f365e79-original-pull-secret") pod "global-pull-secret-syncer-4x4z6" (UID: "2639225a-fbee-4ebd-b36f-7ad47f365e79") : object "kube-system"/"original-pull-secret" not registered Jul 17 07:12:08.676120 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:08.675887 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/deb65c16-2d4c-40f0-aa9c-4399304b7222-metrics-certs podName:deb65c16-2d4c-40f0-aa9c-4399304b7222 nodeName:}" failed. No retries permitted until 2026-07-17 07:12:40.675881422 +0000 UTC m=+65.160570378 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/deb65c16-2d4c-40f0-aa9c-4399304b7222-metrics-certs") pod "network-metrics-daemon-rvx7b" (UID: "deb65c16-2d4c-40f0-aa9c-4399304b7222") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 07:12:08.776602 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:08.776570 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wmqr\" (UniqueName: \"kubernetes.io/projected/ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376-kube-api-access-9wmqr\") pod \"network-check-target-dtjs7\" (UID: \"ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376\") " pod="openshift-network-diagnostics/network-check-target-dtjs7" Jul 17 07:12:08.776748 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:08.776683 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 17 07:12:08.776748 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:08.776698 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 17 07:12:08.776748 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:08.776706 2568 projected.go:196] Error preparing data for projected volume kube-api-access-9wmqr for pod openshift-network-diagnostics/network-check-target-dtjs7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 07:12:08.776837 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:08.776751 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376-kube-api-access-9wmqr podName:ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376 nodeName:}" failed. No retries permitted until 2026-07-17 07:12:40.77673907 +0000 UTC m=+65.261428021 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-9wmqr" (UniqueName: "kubernetes.io/projected/ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376-kube-api-access-9wmqr") pod "network-check-target-dtjs7" (UID: "ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 07:12:08.994019 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:08.993942 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rvx7b" Jul 17 07:12:08.994144 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:08.994048 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rvx7b" podUID="deb65c16-2d4c-40f0-aa9c-4399304b7222" Jul 17 07:12:09.268033 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:09.267955 2568 generic.go:356] "Generic (PLEG): container finished" podID="4fbcebdd-a629-4d9a-9ba1-39e350bfa786" containerID="299c93549eede4d0802f32d80fd72bf0ec772f9f35a974b23a7e1f2fb892a93e" exitCode=0 Jul 17 07:12:09.268033 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:09.268011 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z2q9x" event={"ID":"4fbcebdd-a629-4d9a-9ba1-39e350bfa786","Type":"ContainerDied","Data":"299c93549eede4d0802f32d80fd72bf0ec772f9f35a974b23a7e1f2fb892a93e"} Jul 17 07:12:09.993964 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:09.993883 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dtjs7" Jul 17 07:12:09.993964 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:09.993892 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4x4z6" Jul 17 07:12:09.994429 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:09.994032 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dtjs7" podUID="ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376" Jul 17 07:12:09.994429 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:09.994132 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4x4z6" podUID="2639225a-fbee-4ebd-b36f-7ad47f365e79" Jul 17 07:12:10.993686 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:10.993650 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rvx7b" Jul 17 07:12:10.993877 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:10.993777 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rvx7b" podUID="deb65c16-2d4c-40f0-aa9c-4399304b7222" Jul 17 07:12:11.253981 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.253909 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-233.ec2.internal" event="NodeReady" Jul 17 07:12:11.254421 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.254062 2568 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 17 07:12:11.290344 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.290314 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6675666ffd-gnf79"] Jul 17 07:12:11.294643 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.294620 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6675666ffd-gnf79" Jul 17 07:12:11.297183 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.297161 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 17 07:12:11.297379 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.297208 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-lwnzz\"" Jul 17 07:12:11.297526 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.297440 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 17 07:12:11.297593 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.297564 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 17 07:12:11.302041 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.301739 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-vc255"] Jul 17 07:12:11.303582 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.303404 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 17 07:12:11.305708 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.305693 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-2n5zd"] Jul 17 07:12:11.305844 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.305828 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vc255" Jul 17 07:12:11.307930 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.307908 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-kjdqb\"" Jul 17 07:12:11.308205 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.308190 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 17 07:12:11.308440 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.308424 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 17 07:12:11.309515 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.309495 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6675666ffd-gnf79"] Jul 17 07:12:11.309602 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.309589 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2n5zd" Jul 17 07:12:11.311722 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.311700 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 17 07:12:11.311976 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.311958 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-z4klw\"" Jul 17 07:12:11.312073 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.312056 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 17 07:12:11.312355 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.312330 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 17 07:12:11.316176 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.316158 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2n5zd"] Jul 17 07:12:11.321989 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.321968 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vc255"] Jul 17 07:12:11.397223 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.397184 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0161fccb-b49c-4592-8a96-b79d237fe749-image-registry-private-configuration\") pod \"image-registry-6675666ffd-gnf79\" (UID: \"0161fccb-b49c-4592-8a96-b79d237fe749\") " pod="openshift-image-registry/image-registry-6675666ffd-gnf79" Jul 17 07:12:11.397423 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.397238 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kndrl\" (UniqueName: \"kubernetes.io/projected/0161fccb-b49c-4592-8a96-b79d237fe749-kube-api-access-kndrl\") pod \"image-registry-6675666ffd-gnf79\" (UID: \"0161fccb-b49c-4592-8a96-b79d237fe749\") " pod="openshift-image-registry/image-registry-6675666ffd-gnf79" Jul 17 07:12:11.397423 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.397332 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2df1cbe7-b30d-450f-b2e8-c3f4e10d00d5-cert\") pod \"ingress-canary-2n5zd\" (UID: \"2df1cbe7-b30d-450f-b2e8-c3f4e10d00d5\") " pod="openshift-ingress-canary/ingress-canary-2n5zd" Jul 17 07:12:11.397423 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.397371 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0161fccb-b49c-4592-8a96-b79d237fe749-registry-certificates\") pod \"image-registry-6675666ffd-gnf79\" (UID: \"0161fccb-b49c-4592-8a96-b79d237fe749\") " pod="openshift-image-registry/image-registry-6675666ffd-gnf79" Jul 17 07:12:11.397423 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.397395 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b1d8b7b7-173b-4e59-acce-7fbe2b1f5465-config-volume\") pod \"dns-default-vc255\" (UID: \"b1d8b7b7-173b-4e59-acce-7fbe2b1f5465\") " pod="openshift-dns/dns-default-vc255" Jul 17 07:12:11.397592 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.397446 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0161fccb-b49c-4592-8a96-b79d237fe749-registry-tls\") pod \"image-registry-6675666ffd-gnf79\" (UID: \"0161fccb-b49c-4592-8a96-b79d237fe749\") " pod="openshift-image-registry/image-registry-6675666ffd-gnf79" Jul 17 07:12:11.397592 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.397476 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b46lx\" (UniqueName: \"kubernetes.io/projected/b1d8b7b7-173b-4e59-acce-7fbe2b1f5465-kube-api-access-b46lx\") pod \"dns-default-vc255\" (UID: \"b1d8b7b7-173b-4e59-acce-7fbe2b1f5465\") " pod="openshift-dns/dns-default-vc255" Jul 17 07:12:11.397592 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.397507 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0161fccb-b49c-4592-8a96-b79d237fe749-bound-sa-token\") pod \"image-registry-6675666ffd-gnf79\" (UID: \"0161fccb-b49c-4592-8a96-b79d237fe749\") " pod="openshift-image-registry/image-registry-6675666ffd-gnf79" Jul 17 07:12:11.397592 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.397534 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kq9d\" (UniqueName: \"kubernetes.io/projected/2df1cbe7-b30d-450f-b2e8-c3f4e10d00d5-kube-api-access-4kq9d\") pod \"ingress-canary-2n5zd\" (UID: \"2df1cbe7-b30d-450f-b2e8-c3f4e10d00d5\") " pod="openshift-ingress-canary/ingress-canary-2n5zd" Jul 17 07:12:11.397592 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.397561 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0161fccb-b49c-4592-8a96-b79d237fe749-ca-trust-extracted\") pod \"image-registry-6675666ffd-gnf79\" (UID: \"0161fccb-b49c-4592-8a96-b79d237fe749\") " pod="openshift-image-registry/image-registry-6675666ffd-gnf79" Jul 17 07:12:11.397592 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.397587 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0161fccb-b49c-4592-8a96-b79d237fe749-installation-pull-secrets\") pod \"image-registry-6675666ffd-gnf79\" (UID: \"0161fccb-b49c-4592-8a96-b79d237fe749\") " pod="openshift-image-registry/image-registry-6675666ffd-gnf79" Jul 17 07:12:11.397826 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.397645 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b1d8b7b7-173b-4e59-acce-7fbe2b1f5465-metrics-tls\") pod \"dns-default-vc255\" (UID: \"b1d8b7b7-173b-4e59-acce-7fbe2b1f5465\") " pod="openshift-dns/dns-default-vc255" Jul 17 07:12:11.397826 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.397676 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b1d8b7b7-173b-4e59-acce-7fbe2b1f5465-tmp-dir\") pod \"dns-default-vc255\" (UID: \"b1d8b7b7-173b-4e59-acce-7fbe2b1f5465\") " pod="openshift-dns/dns-default-vc255" Jul 17 07:12:11.397826 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.397724 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0161fccb-b49c-4592-8a96-b79d237fe749-trusted-ca\") pod \"image-registry-6675666ffd-gnf79\" (UID: \"0161fccb-b49c-4592-8a96-b79d237fe749\") " pod="openshift-image-registry/image-registry-6675666ffd-gnf79" Jul 17 07:12:11.499033 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.498994 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kq9d\" (UniqueName: \"kubernetes.io/projected/2df1cbe7-b30d-450f-b2e8-c3f4e10d00d5-kube-api-access-4kq9d\") pod \"ingress-canary-2n5zd\" (UID: \"2df1cbe7-b30d-450f-b2e8-c3f4e10d00d5\") " pod="openshift-ingress-canary/ingress-canary-2n5zd" Jul 17 07:12:11.499033 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.499038 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0161fccb-b49c-4592-8a96-b79d237fe749-ca-trust-extracted\") pod \"image-registry-6675666ffd-gnf79\" (UID: \"0161fccb-b49c-4592-8a96-b79d237fe749\") " pod="openshift-image-registry/image-registry-6675666ffd-gnf79" Jul 17 07:12:11.499273 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.499063 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0161fccb-b49c-4592-8a96-b79d237fe749-installation-pull-secrets\") pod \"image-registry-6675666ffd-gnf79\" (UID: \"0161fccb-b49c-4592-8a96-b79d237fe749\") " pod="openshift-image-registry/image-registry-6675666ffd-gnf79" Jul 17 07:12:11.499273 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.499111 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b1d8b7b7-173b-4e59-acce-7fbe2b1f5465-metrics-tls\") pod \"dns-default-vc255\" (UID: \"b1d8b7b7-173b-4e59-acce-7fbe2b1f5465\") " pod="openshift-dns/dns-default-vc255" Jul 17 07:12:11.499273 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.499132 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b1d8b7b7-173b-4e59-acce-7fbe2b1f5465-tmp-dir\") pod \"dns-default-vc255\" (UID: \"b1d8b7b7-173b-4e59-acce-7fbe2b1f5465\") " pod="openshift-dns/dns-default-vc255" Jul 17 07:12:11.499273 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.499159 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0161fccb-b49c-4592-8a96-b79d237fe749-trusted-ca\") pod \"image-registry-6675666ffd-gnf79\" (UID: \"0161fccb-b49c-4592-8a96-b79d237fe749\") " pod="openshift-image-registry/image-registry-6675666ffd-gnf79" Jul 17 07:12:11.499273 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.499189 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0161fccb-b49c-4592-8a96-b79d237fe749-image-registry-private-configuration\") pod \"image-registry-6675666ffd-gnf79\" (UID: \"0161fccb-b49c-4592-8a96-b79d237fe749\") " pod="openshift-image-registry/image-registry-6675666ffd-gnf79" Jul 17 07:12:11.499273 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.499218 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kndrl\" (UniqueName: \"kubernetes.io/projected/0161fccb-b49c-4592-8a96-b79d237fe749-kube-api-access-kndrl\") pod \"image-registry-6675666ffd-gnf79\" (UID: \"0161fccb-b49c-4592-8a96-b79d237fe749\") " pod="openshift-image-registry/image-registry-6675666ffd-gnf79" Jul 17 07:12:11.499273 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.499257 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2df1cbe7-b30d-450f-b2e8-c3f4e10d00d5-cert\") pod \"ingress-canary-2n5zd\" (UID: \"2df1cbe7-b30d-450f-b2e8-c3f4e10d00d5\") " pod="openshift-ingress-canary/ingress-canary-2n5zd" Jul 17 07:12:11.499636 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.499300 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0161fccb-b49c-4592-8a96-b79d237fe749-registry-certificates\") pod \"image-registry-6675666ffd-gnf79\" (UID: \"0161fccb-b49c-4592-8a96-b79d237fe749\") " pod="openshift-image-registry/image-registry-6675666ffd-gnf79" Jul 17 07:12:11.499636 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:11.499306 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 17 07:12:11.499636 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.499325 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b1d8b7b7-173b-4e59-acce-7fbe2b1f5465-config-volume\") pod \"dns-default-vc255\" (UID: \"b1d8b7b7-173b-4e59-acce-7fbe2b1f5465\") " pod="openshift-dns/dns-default-vc255" Jul 17 07:12:11.499636 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.499366 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0161fccb-b49c-4592-8a96-b79d237fe749-registry-tls\") pod \"image-registry-6675666ffd-gnf79\" (UID: \"0161fccb-b49c-4592-8a96-b79d237fe749\") " pod="openshift-image-registry/image-registry-6675666ffd-gnf79" Jul 17 07:12:11.499636 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.499391 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b46lx\" (UniqueName: \"kubernetes.io/projected/b1d8b7b7-173b-4e59-acce-7fbe2b1f5465-kube-api-access-b46lx\") pod \"dns-default-vc255\" (UID: \"b1d8b7b7-173b-4e59-acce-7fbe2b1f5465\") " pod="openshift-dns/dns-default-vc255" Jul 17 07:12:11.499636 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.499421 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0161fccb-b49c-4592-8a96-b79d237fe749-bound-sa-token\") pod \"image-registry-6675666ffd-gnf79\" (UID: \"0161fccb-b49c-4592-8a96-b79d237fe749\") " pod="openshift-image-registry/image-registry-6675666ffd-gnf79" Jul 17 07:12:11.499636 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:11.499517 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1d8b7b7-173b-4e59-acce-7fbe2b1f5465-metrics-tls podName:b1d8b7b7-173b-4e59-acce-7fbe2b1f5465 nodeName:}" failed. No retries permitted until 2026-07-17 07:12:11.999495378 +0000 UTC m=+36.484184335 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b1d8b7b7-173b-4e59-acce-7fbe2b1f5465-metrics-tls") pod "dns-default-vc255" (UID: "b1d8b7b7-173b-4e59-acce-7fbe2b1f5465") : secret "dns-default-metrics-tls" not found Jul 17 07:12:11.499636 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:11.499592 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 17 07:12:11.499636 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:11.499604 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6675666ffd-gnf79: secret "image-registry-tls" not found Jul 17 07:12:11.499636 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:11.499634 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0161fccb-b49c-4592-8a96-b79d237fe749-registry-tls podName:0161fccb-b49c-4592-8a96-b79d237fe749 nodeName:}" failed. No retries permitted until 2026-07-17 07:12:11.999626979 +0000 UTC m=+36.484315967 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0161fccb-b49c-4592-8a96-b79d237fe749-registry-tls") pod "image-registry-6675666ffd-gnf79" (UID: "0161fccb-b49c-4592-8a96-b79d237fe749") : secret "image-registry-tls" not found Jul 17 07:12:11.500189 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.499901 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b1d8b7b7-173b-4e59-acce-7fbe2b1f5465-tmp-dir\") pod \"dns-default-vc255\" (UID: \"b1d8b7b7-173b-4e59-acce-7fbe2b1f5465\") " pod="openshift-dns/dns-default-vc255" Jul 17 07:12:11.500189 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.500084 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b1d8b7b7-173b-4e59-acce-7fbe2b1f5465-config-volume\") pod \"dns-default-vc255\" (UID: \"b1d8b7b7-173b-4e59-acce-7fbe2b1f5465\") " pod="openshift-dns/dns-default-vc255" Jul 17 07:12:11.500323 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.500220 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0161fccb-b49c-4592-8a96-b79d237fe749-registry-certificates\") pod \"image-registry-6675666ffd-gnf79\" (UID: \"0161fccb-b49c-4592-8a96-b79d237fe749\") " pod="openshift-image-registry/image-registry-6675666ffd-gnf79" Jul 17 07:12:11.500616 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.500539 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0161fccb-b49c-4592-8a96-b79d237fe749-ca-trust-extracted\") pod \"image-registry-6675666ffd-gnf79\" (UID: \"0161fccb-b49c-4592-8a96-b79d237fe749\") " pod="openshift-image-registry/image-registry-6675666ffd-gnf79" Jul 17 07:12:11.500774 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.500638 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0161fccb-b49c-4592-8a96-b79d237fe749-trusted-ca\") pod \"image-registry-6675666ffd-gnf79\" (UID: \"0161fccb-b49c-4592-8a96-b79d237fe749\") " pod="openshift-image-registry/image-registry-6675666ffd-gnf79" Jul 17 07:12:11.503810 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.503778 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0161fccb-b49c-4592-8a96-b79d237fe749-image-registry-private-configuration\") pod \"image-registry-6675666ffd-gnf79\" (UID: \"0161fccb-b49c-4592-8a96-b79d237fe749\") " pod="openshift-image-registry/image-registry-6675666ffd-gnf79" Jul 17 07:12:11.503810 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.503794 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0161fccb-b49c-4592-8a96-b79d237fe749-installation-pull-secrets\") pod \"image-registry-6675666ffd-gnf79\" (UID: \"0161fccb-b49c-4592-8a96-b79d237fe749\") " pod="openshift-image-registry/image-registry-6675666ffd-gnf79" Jul 17 07:12:11.503810 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.503806 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2df1cbe7-b30d-450f-b2e8-c3f4e10d00d5-cert\") pod \"ingress-canary-2n5zd\" (UID: \"2df1cbe7-b30d-450f-b2e8-c3f4e10d00d5\") " pod="openshift-ingress-canary/ingress-canary-2n5zd" Jul 17 07:12:11.510676 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.510619 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b46lx\" (UniqueName: \"kubernetes.io/projected/b1d8b7b7-173b-4e59-acce-7fbe2b1f5465-kube-api-access-b46lx\") pod \"dns-default-vc255\" (UID: \"b1d8b7b7-173b-4e59-acce-7fbe2b1f5465\") " pod="openshift-dns/dns-default-vc255" Jul 17 07:12:11.511115 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.511092 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kq9d\" (UniqueName: \"kubernetes.io/projected/2df1cbe7-b30d-450f-b2e8-c3f4e10d00d5-kube-api-access-4kq9d\") pod \"ingress-canary-2n5zd\" (UID: \"2df1cbe7-b30d-450f-b2e8-c3f4e10d00d5\") " pod="openshift-ingress-canary/ingress-canary-2n5zd" Jul 17 07:12:11.511584 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.511561 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0161fccb-b49c-4592-8a96-b79d237fe749-bound-sa-token\") pod \"image-registry-6675666ffd-gnf79\" (UID: \"0161fccb-b49c-4592-8a96-b79d237fe749\") " pod="openshift-image-registry/image-registry-6675666ffd-gnf79" Jul 17 07:12:11.512227 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.512200 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kndrl\" (UniqueName: \"kubernetes.io/projected/0161fccb-b49c-4592-8a96-b79d237fe749-kube-api-access-kndrl\") pod \"image-registry-6675666ffd-gnf79\" (UID: \"0161fccb-b49c-4592-8a96-b79d237fe749\") " pod="openshift-image-registry/image-registry-6675666ffd-gnf79" Jul 17 07:12:11.622725 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.622687 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2n5zd" Jul 17 07:12:11.776938 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.776857 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2n5zd"] Jul 17 07:12:11.782738 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:12:11.782712 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2df1cbe7_b30d_450f_b2e8_c3f4e10d00d5.slice/crio-daa60d7811444b47fe76ccfcc8b1419eb33254797ae81fd70476f66826805fec WatchSource:0}: Error finding container daa60d7811444b47fe76ccfcc8b1419eb33254797ae81fd70476f66826805fec: Status 404 returned error can't find the container with id daa60d7811444b47fe76ccfcc8b1419eb33254797ae81fd70476f66826805fec Jul 17 07:12:11.994311 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.994264 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dtjs7" Jul 17 07:12:11.994456 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.994323 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4x4z6" Jul 17 07:12:11.996873 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.996852 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 17 07:12:11.997001 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.996940 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 17 07:12:11.997001 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.996940 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-c28tv\"" Jul 17 07:12:11.997001 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:11.996975 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 17 07:12:12.003935 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:12.003908 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b1d8b7b7-173b-4e59-acce-7fbe2b1f5465-metrics-tls\") pod \"dns-default-vc255\" (UID: \"b1d8b7b7-173b-4e59-acce-7fbe2b1f5465\") " pod="openshift-dns/dns-default-vc255" Jul 17 07:12:12.004034 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:12.004019 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0161fccb-b49c-4592-8a96-b79d237fe749-registry-tls\") pod \"image-registry-6675666ffd-gnf79\" (UID: \"0161fccb-b49c-4592-8a96-b79d237fe749\") " pod="openshift-image-registry/image-registry-6675666ffd-gnf79" Jul 17 07:12:12.004133 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:12.004098 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 17 07:12:12.004243 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:12.004135 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 17 07:12:12.004243 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:12.004153 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6675666ffd-gnf79: secret "image-registry-tls" not found Jul 17 07:12:12.004243 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:12.004173 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1d8b7b7-173b-4e59-acce-7fbe2b1f5465-metrics-tls podName:b1d8b7b7-173b-4e59-acce-7fbe2b1f5465 nodeName:}" failed. No retries permitted until 2026-07-17 07:12:13.004153412 +0000 UTC m=+37.488842366 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b1d8b7b7-173b-4e59-acce-7fbe2b1f5465-metrics-tls") pod "dns-default-vc255" (UID: "b1d8b7b7-173b-4e59-acce-7fbe2b1f5465") : secret "dns-default-metrics-tls" not found Jul 17 07:12:12.004243 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:12.004204 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0161fccb-b49c-4592-8a96-b79d237fe749-registry-tls podName:0161fccb-b49c-4592-8a96-b79d237fe749 nodeName:}" failed. No retries permitted until 2026-07-17 07:12:13.00418445 +0000 UTC m=+37.488873405 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0161fccb-b49c-4592-8a96-b79d237fe749-registry-tls") pod "image-registry-6675666ffd-gnf79" (UID: "0161fccb-b49c-4592-8a96-b79d237fe749") : secret "image-registry-tls" not found Jul 17 07:12:12.275431 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:12.275343 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2n5zd" event={"ID":"2df1cbe7-b30d-450f-b2e8-c3f4e10d00d5","Type":"ContainerStarted","Data":"daa60d7811444b47fe76ccfcc8b1419eb33254797ae81fd70476f66826805fec"} Jul 17 07:12:12.993994 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:12.993958 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rvx7b" Jul 17 07:12:12.996535 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:12.996508 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 17 07:12:12.996535 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:12.996521 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-s4jgz\"" Jul 17 07:12:13.013216 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:13.013196 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0161fccb-b49c-4592-8a96-b79d237fe749-registry-tls\") pod \"image-registry-6675666ffd-gnf79\" (UID: \"0161fccb-b49c-4592-8a96-b79d237fe749\") " pod="openshift-image-registry/image-registry-6675666ffd-gnf79" Jul 17 07:12:13.013348 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:13.013257 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b1d8b7b7-173b-4e59-acce-7fbe2b1f5465-metrics-tls\") pod \"dns-default-vc255\" (UID: \"b1d8b7b7-173b-4e59-acce-7fbe2b1f5465\") " pod="openshift-dns/dns-default-vc255" Jul 17 07:12:13.013407 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:13.013349 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 17 07:12:13.013407 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:13.013363 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6675666ffd-gnf79: secret "image-registry-tls" not found Jul 17 07:12:13.013489 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:13.013399 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 17 07:12:13.013489 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:13.013414 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0161fccb-b49c-4592-8a96-b79d237fe749-registry-tls podName:0161fccb-b49c-4592-8a96-b79d237fe749 nodeName:}" failed. No retries permitted until 2026-07-17 07:12:15.013400589 +0000 UTC m=+39.498089557 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0161fccb-b49c-4592-8a96-b79d237fe749-registry-tls") pod "image-registry-6675666ffd-gnf79" (UID: "0161fccb-b49c-4592-8a96-b79d237fe749") : secret "image-registry-tls" not found Jul 17 07:12:13.013489 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:13.013456 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1d8b7b7-173b-4e59-acce-7fbe2b1f5465-metrics-tls podName:b1d8b7b7-173b-4e59-acce-7fbe2b1f5465 nodeName:}" failed. No retries permitted until 2026-07-17 07:12:15.013440223 +0000 UTC m=+39.498129181 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b1d8b7b7-173b-4e59-acce-7fbe2b1f5465-metrics-tls") pod "dns-default-vc255" (UID: "b1d8b7b7-173b-4e59-acce-7fbe2b1f5465") : secret "dns-default-metrics-tls" not found Jul 17 07:12:14.284241 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:14.283988 2568 generic.go:356] "Generic (PLEG): container finished" podID="4fbcebdd-a629-4d9a-9ba1-39e350bfa786" containerID="17d13f39d7d6554648dc89736dccb136afbefd5215e40e6ce652eb5685825bff" exitCode=0 Jul 17 07:12:14.284637 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:14.284067 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z2q9x" event={"ID":"4fbcebdd-a629-4d9a-9ba1-39e350bfa786","Type":"ContainerDied","Data":"17d13f39d7d6554648dc89736dccb136afbefd5215e40e6ce652eb5685825bff"} Jul 17 07:12:15.030681 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:15.030583 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b1d8b7b7-173b-4e59-acce-7fbe2b1f5465-metrics-tls\") pod \"dns-default-vc255\" (UID: \"b1d8b7b7-173b-4e59-acce-7fbe2b1f5465\") " pod="openshift-dns/dns-default-vc255" Jul 17 07:12:15.030863 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:15.030705 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0161fccb-b49c-4592-8a96-b79d237fe749-registry-tls\") pod \"image-registry-6675666ffd-gnf79\" (UID: \"0161fccb-b49c-4592-8a96-b79d237fe749\") " pod="openshift-image-registry/image-registry-6675666ffd-gnf79" Jul 17 07:12:15.030863 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:15.030755 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 17 07:12:15.030863 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:15.030831 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 17 07:12:15.030863 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:15.030842 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1d8b7b7-173b-4e59-acce-7fbe2b1f5465-metrics-tls podName:b1d8b7b7-173b-4e59-acce-7fbe2b1f5465 nodeName:}" failed. No retries permitted until 2026-07-17 07:12:19.030818883 +0000 UTC m=+43.515507837 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b1d8b7b7-173b-4e59-acce-7fbe2b1f5465-metrics-tls") pod "dns-default-vc255" (UID: "b1d8b7b7-173b-4e59-acce-7fbe2b1f5465") : secret "dns-default-metrics-tls" not found Jul 17 07:12:15.030863 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:15.030845 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6675666ffd-gnf79: secret "image-registry-tls" not found Jul 17 07:12:15.031070 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:15.030887 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0161fccb-b49c-4592-8a96-b79d237fe749-registry-tls podName:0161fccb-b49c-4592-8a96-b79d237fe749 nodeName:}" failed. No retries permitted until 2026-07-17 07:12:19.030876948 +0000 UTC m=+43.515565900 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0161fccb-b49c-4592-8a96-b79d237fe749-registry-tls") pod "image-registry-6675666ffd-gnf79" (UID: "0161fccb-b49c-4592-8a96-b79d237fe749") : secret "image-registry-tls" not found Jul 17 07:12:15.291457 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:15.291367 2568 generic.go:356] "Generic (PLEG): container finished" podID="4fbcebdd-a629-4d9a-9ba1-39e350bfa786" containerID="59fd9f83fbf37c27dfcf35ff3d1ea4f847d4d8ad666d4e6a44b456f57053de61" exitCode=0 Jul 17 07:12:15.291457 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:15.291435 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z2q9x" event={"ID":"4fbcebdd-a629-4d9a-9ba1-39e350bfa786","Type":"ContainerDied","Data":"59fd9f83fbf37c27dfcf35ff3d1ea4f847d4d8ad666d4e6a44b456f57053de61"} Jul 17 07:12:16.295480 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.295397 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2n5zd" event={"ID":"2df1cbe7-b30d-450f-b2e8-c3f4e10d00d5","Type":"ContainerStarted","Data":"4b02eab1588e94956724203687eb9a037619045b08746df4844147f3fc3ea45b"} Jul 17 07:12:16.299122 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.299095 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z2q9x" event={"ID":"4fbcebdd-a629-4d9a-9ba1-39e350bfa786","Type":"ContainerStarted","Data":"c987588abc74a1c269b59db82cfc00b4d806ae4e22790bc66b5c2e2ca7b54030"} Jul 17 07:12:16.312476 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.312431 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-2n5zd" podStartSLOduration=1.090878152 podStartE2EDuration="5.312416002s" totalImagesPullingTime="4.22153785s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 07:12:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:12:16.311897819 +0000 UTC m=+40.796586790" watchObservedRunningTime="2026-07-17 07:12:16.312416002 +0000 UTC m=+40.797104980" Jul 17 07:12:16.333981 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.333944 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-z2q9x" podStartSLOduration=8.635274563 podStartE2EDuration="40.333931711s" totalImagesPullingTime="31.698657148s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 07:11:36 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:12:16.332715118 +0000 UTC m=+40.817404092" watchObservedRunningTime="2026-07-17 07:12:16.333931711 +0000 UTC m=+40.818620682" Jul 17 07:12:16.658633 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.658558 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-xpxv5"] Jul 17 07:12:16.680245 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.680218 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-z6kdb"] Jul 17 07:12:16.680398 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.680379 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-xpxv5" Jul 17 07:12:16.682682 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.682658 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-6ppbq\"" Jul 17 07:12:16.682823 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.682710 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 17 07:12:16.682823 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.682734 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 17 07:12:16.682823 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.682820 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 17 07:12:16.682959 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.682947 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 17 07:12:16.687953 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.687934 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 17 07:12:16.700215 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.700194 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-6784546c7b-l47vf"] Jul 17 07:12:16.700339 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.700323 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-z6kdb" Jul 17 07:12:16.702654 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.702637 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 17 07:12:16.702857 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.702832 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 17 07:12:16.702908 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.702869 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-pxg4x\"" Jul 17 07:12:16.702908 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.702896 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 17 07:12:16.722267 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.722248 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-xpxv5"] Jul 17 07:12:16.722383 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.722273 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-z6kdb"] Jul 17 07:12:16.722383 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.722308 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6784546c7b-l47vf"] Jul 17 07:12:16.722483 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.722379 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6784546c7b-l47vf" Jul 17 07:12:16.724887 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.724860 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 17 07:12:16.724887 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.724873 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 17 07:12:16.725106 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.725092 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-djw4q\"" Jul 17 07:12:16.725160 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.725122 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 17 07:12:16.725401 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.725381 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 17 07:12:16.725493 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.725390 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 17 07:12:16.725563 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.725441 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 17 07:12:16.758128 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.758105 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-xgtz8"] Jul 17 07:12:16.780445 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.780420 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xr6pg"] Jul 17 07:12:16.781438 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.781418 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-xgtz8" Jul 17 07:12:16.784437 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.784416 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 17 07:12:16.784573 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.784482 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 17 07:12:16.784746 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.784726 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-rghj4\"" Jul 17 07:12:16.804339 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.804316 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-jv9pk"] Jul 17 07:12:16.804455 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.804438 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xr6pg" Jul 17 07:12:16.807970 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.807948 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 17 07:12:16.808053 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.807970 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-78zzm\"" Jul 17 07:12:16.808105 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.808087 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 17 07:12:16.808141 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.808124 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 17 07:12:16.808267 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.808236 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 17 07:12:16.826942 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.826921 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-xgtz8"] Jul 17 07:12:16.827018 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.826946 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xr6pg"] Jul 17 07:12:16.827018 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.826960 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-jv9pk"] Jul 17 07:12:16.827084 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.827062 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-jv9pk" Jul 17 07:12:16.829473 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.829457 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 17 07:12:16.829560 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.829543 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-wcwsr\"" Jul 17 07:12:16.829774 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.829756 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 17 07:12:16.829845 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.829785 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 17 07:12:16.829894 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.829886 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 17 07:12:16.834550 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.834534 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 17 07:12:16.845093 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.845073 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/27538d1e-992b-4651-a2d2-7a7ecf442806-serving-cert\") pod \"console-operator-565b7b569d-xpxv5\" (UID: \"27538d1e-992b-4651-a2d2-7a7ecf442806\") " pod="openshift-console-operator/console-operator-565b7b569d-xpxv5" Jul 17 07:12:16.845177 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.845101 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2b81b5e5-a313-4e9c-87e6-4b0242f46a78-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-z6kdb\" (UID: \"2b81b5e5-a313-4e9c-87e6-4b0242f46a78\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-z6kdb" Jul 17 07:12:16.845177 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.845120 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/254d445f-a7c6-49e6-a9a6-51b738ccdab2-stats-auth\") pod \"router-default-6784546c7b-l47vf\" (UID: \"254d445f-a7c6-49e6-a9a6-51b738ccdab2\") " pod="openshift-ingress/router-default-6784546c7b-l47vf" Jul 17 07:12:16.845244 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.845188 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/254d445f-a7c6-49e6-a9a6-51b738ccdab2-service-ca-bundle\") pod \"router-default-6784546c7b-l47vf\" (UID: \"254d445f-a7c6-49e6-a9a6-51b738ccdab2\") " pod="openshift-ingress/router-default-6784546c7b-l47vf" Jul 17 07:12:16.845244 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.845232 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/27538d1e-992b-4651-a2d2-7a7ecf442806-config\") pod \"console-operator-565b7b569d-xpxv5\" (UID: \"27538d1e-992b-4651-a2d2-7a7ecf442806\") " pod="openshift-console-operator/console-operator-565b7b569d-xpxv5" Jul 17 07:12:16.845326 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.845250 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/254d445f-a7c6-49e6-a9a6-51b738ccdab2-metrics-certs\") pod \"router-default-6784546c7b-l47vf\" (UID: \"254d445f-a7c6-49e6-a9a6-51b738ccdab2\") " pod="openshift-ingress/router-default-6784546c7b-l47vf" Jul 17 07:12:16.845326 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.845270 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/254d445f-a7c6-49e6-a9a6-51b738ccdab2-default-certificate\") pod \"router-default-6784546c7b-l47vf\" (UID: \"254d445f-a7c6-49e6-a9a6-51b738ccdab2\") " pod="openshift-ingress/router-default-6784546c7b-l47vf" Jul 17 07:12:16.845407 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.845334 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/27538d1e-992b-4651-a2d2-7a7ecf442806-trusted-ca\") pod \"console-operator-565b7b569d-xpxv5\" (UID: \"27538d1e-992b-4651-a2d2-7a7ecf442806\") " pod="openshift-console-operator/console-operator-565b7b569d-xpxv5" Jul 17 07:12:16.845451 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.845418 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cckw7\" (UniqueName: \"kubernetes.io/projected/2b81b5e5-a313-4e9c-87e6-4b0242f46a78-kube-api-access-cckw7\") pod \"cluster-samples-operator-79b7c55f87-z6kdb\" (UID: \"2b81b5e5-a313-4e9c-87e6-4b0242f46a78\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-z6kdb" Jul 17 07:12:16.845487 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.845463 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrqmd\" (UniqueName: \"kubernetes.io/projected/254d445f-a7c6-49e6-a9a6-51b738ccdab2-kube-api-access-lrqmd\") pod \"router-default-6784546c7b-l47vf\" (UID: \"254d445f-a7c6-49e6-a9a6-51b738ccdab2\") " pod="openshift-ingress/router-default-6784546c7b-l47vf" Jul 17 07:12:16.845528 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.845500 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rk7b\" (UniqueName: \"kubernetes.io/projected/27538d1e-992b-4651-a2d2-7a7ecf442806-kube-api-access-2rk7b\") pod \"console-operator-565b7b569d-xpxv5\" (UID: \"27538d1e-992b-4651-a2d2-7a7ecf442806\") " pod="openshift-console-operator/console-operator-565b7b569d-xpxv5" Jul 17 07:12:16.946089 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.946027 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/254d445f-a7c6-49e6-a9a6-51b738ccdab2-metrics-certs\") pod \"router-default-6784546c7b-l47vf\" (UID: \"254d445f-a7c6-49e6-a9a6-51b738ccdab2\") " pod="openshift-ingress/router-default-6784546c7b-l47vf" Jul 17 07:12:16.946198 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.946098 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a3b3827-cd5d-4292-8ce8-4e1c17c4c1e4-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-jv9pk\" (UID: \"2a3b3827-cd5d-4292-8ce8-4e1c17c4c1e4\") " pod="openshift-insights/insights-operator-555d9d7885-jv9pk" Jul 17 07:12:16.946198 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.946127 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a3b3827-cd5d-4292-8ce8-4e1c17c4c1e4-service-ca-bundle\") pod \"insights-operator-555d9d7885-jv9pk\" (UID: \"2a3b3827-cd5d-4292-8ce8-4e1c17c4c1e4\") " pod="openshift-insights/insights-operator-555d9d7885-jv9pk" Jul 17 07:12:16.946198 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.946150 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k72rq\" (UniqueName: \"kubernetes.io/projected/2a3b3827-cd5d-4292-8ce8-4e1c17c4c1e4-kube-api-access-k72rq\") pod \"insights-operator-555d9d7885-jv9pk\" (UID: \"2a3b3827-cd5d-4292-8ce8-4e1c17c4c1e4\") " pod="openshift-insights/insights-operator-555d9d7885-jv9pk" Jul 17 07:12:16.946198 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:16.946165 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 17 07:12:16.946198 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.946178 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rk7b\" (UniqueName: \"kubernetes.io/projected/27538d1e-992b-4651-a2d2-7a7ecf442806-kube-api-access-2rk7b\") pod \"console-operator-565b7b569d-xpxv5\" (UID: \"27538d1e-992b-4651-a2d2-7a7ecf442806\") " pod="openshift-console-operator/console-operator-565b7b569d-xpxv5" Jul 17 07:12:16.946444 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:16.946221 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/254d445f-a7c6-49e6-a9a6-51b738ccdab2-metrics-certs podName:254d445f-a7c6-49e6-a9a6-51b738ccdab2 nodeName:}" failed. No retries permitted until 2026-07-17 07:12:17.446202965 +0000 UTC m=+41.930891921 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/254d445f-a7c6-49e6-a9a6-51b738ccdab2-metrics-certs") pod "router-default-6784546c7b-l47vf" (UID: "254d445f-a7c6-49e6-a9a6-51b738ccdab2") : secret "router-metrics-certs-default" not found Jul 17 07:12:16.946444 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.946303 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2b81b5e5-a313-4e9c-87e6-4b0242f46a78-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-z6kdb\" (UID: \"2b81b5e5-a313-4e9c-87e6-4b0242f46a78\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-z6kdb" Jul 17 07:12:16.946444 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.946331 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/254d445f-a7c6-49e6-a9a6-51b738ccdab2-stats-auth\") pod \"router-default-6784546c7b-l47vf\" (UID: \"254d445f-a7c6-49e6-a9a6-51b738ccdab2\") " pod="openshift-ingress/router-default-6784546c7b-l47vf" Jul 17 07:12:16.946444 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.946351 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57csl\" (UniqueName: \"kubernetes.io/projected/6462d06e-f55f-44cc-9913-ef8120a6f4cc-kube-api-access-57csl\") pod \"volume-data-source-validator-78cf4c6f6-xgtz8\" (UID: \"6462d06e-f55f-44cc-9913-ef8120a6f4cc\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-xgtz8" Jul 17 07:12:16.946444 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:16.946367 2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 17 07:12:16.946444 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:16.946433 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b81b5e5-a313-4e9c-87e6-4b0242f46a78-samples-operator-tls podName:2b81b5e5-a313-4e9c-87e6-4b0242f46a78 nodeName:}" failed. No retries permitted until 2026-07-17 07:12:17.446418227 +0000 UTC m=+41.931107191 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2b81b5e5-a313-4e9c-87e6-4b0242f46a78-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-z6kdb" (UID: "2b81b5e5-a313-4e9c-87e6-4b0242f46a78") : secret "samples-operator-tls" not found Jul 17 07:12:16.946686 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.946460 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/254d445f-a7c6-49e6-a9a6-51b738ccdab2-service-ca-bundle\") pod \"router-default-6784546c7b-l47vf\" (UID: \"254d445f-a7c6-49e6-a9a6-51b738ccdab2\") " pod="openshift-ingress/router-default-6784546c7b-l47vf" Jul 17 07:12:16.946686 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.946495 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2a3b3827-cd5d-4292-8ce8-4e1c17c4c1e4-tmp\") pod \"insights-operator-555d9d7885-jv9pk\" (UID: \"2a3b3827-cd5d-4292-8ce8-4e1c17c4c1e4\") " pod="openshift-insights/insights-operator-555d9d7885-jv9pk" Jul 17 07:12:16.946686 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:16.946621 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/254d445f-a7c6-49e6-a9a6-51b738ccdab2-service-ca-bundle podName:254d445f-a7c6-49e6-a9a6-51b738ccdab2 nodeName:}" failed. No retries permitted until 2026-07-17 07:12:17.446604111 +0000 UTC m=+41.931293068 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/254d445f-a7c6-49e6-a9a6-51b738ccdab2-service-ca-bundle") pod "router-default-6784546c7b-l47vf" (UID: "254d445f-a7c6-49e6-a9a6-51b738ccdab2") : configmap references non-existent config key: service-ca.crt Jul 17 07:12:16.946686 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.946662 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2ce00a54-15e9-42b7-b3a8-9e58d1101558-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-xr6pg\" (UID: \"2ce00a54-15e9-42b7-b3a8-9e58d1101558\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xr6pg" Jul 17 07:12:16.946829 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.946696 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/254d445f-a7c6-49e6-a9a6-51b738ccdab2-default-certificate\") pod \"router-default-6784546c7b-l47vf\" (UID: \"254d445f-a7c6-49e6-a9a6-51b738ccdab2\") " pod="openshift-ingress/router-default-6784546c7b-l47vf" Jul 17 07:12:16.946829 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.946718 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/27538d1e-992b-4651-a2d2-7a7ecf442806-trusted-ca\") pod \"console-operator-565b7b569d-xpxv5\" (UID: \"27538d1e-992b-4651-a2d2-7a7ecf442806\") " pod="openshift-console-operator/console-operator-565b7b569d-xpxv5" Jul 17 07:12:16.946829 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.946735 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ce00a54-15e9-42b7-b3a8-9e58d1101558-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-xr6pg\" (UID: \"2ce00a54-15e9-42b7-b3a8-9e58d1101558\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xr6pg" Jul 17 07:12:16.946829 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.946760 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/2a3b3827-cd5d-4292-8ce8-4e1c17c4c1e4-snapshots\") pod \"insights-operator-555d9d7885-jv9pk\" (UID: \"2a3b3827-cd5d-4292-8ce8-4e1c17c4c1e4\") " pod="openshift-insights/insights-operator-555d9d7885-jv9pk" Jul 17 07:12:16.946829 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.946802 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cckw7\" (UniqueName: \"kubernetes.io/projected/2b81b5e5-a313-4e9c-87e6-4b0242f46a78-kube-api-access-cckw7\") pod \"cluster-samples-operator-79b7c55f87-z6kdb\" (UID: \"2b81b5e5-a313-4e9c-87e6-4b0242f46a78\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-z6kdb" Jul 17 07:12:16.947066 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.946843 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrqmd\" (UniqueName: \"kubernetes.io/projected/254d445f-a7c6-49e6-a9a6-51b738ccdab2-kube-api-access-lrqmd\") pod \"router-default-6784546c7b-l47vf\" (UID: \"254d445f-a7c6-49e6-a9a6-51b738ccdab2\") " pod="openshift-ingress/router-default-6784546c7b-l47vf" Jul 17 07:12:16.947066 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.946886 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/27538d1e-992b-4651-a2d2-7a7ecf442806-serving-cert\") pod \"console-operator-565b7b569d-xpxv5\" (UID: \"27538d1e-992b-4651-a2d2-7a7ecf442806\") " pod="openshift-console-operator/console-operator-565b7b569d-xpxv5" Jul 17 07:12:16.947171 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.947052 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/27538d1e-992b-4651-a2d2-7a7ecf442806-config\") pod \"console-operator-565b7b569d-xpxv5\" (UID: \"27538d1e-992b-4651-a2d2-7a7ecf442806\") " pod="openshift-console-operator/console-operator-565b7b569d-xpxv5" Jul 17 07:12:16.947171 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.947112 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2a3b3827-cd5d-4292-8ce8-4e1c17c4c1e4-serving-cert\") pod \"insights-operator-555d9d7885-jv9pk\" (UID: \"2a3b3827-cd5d-4292-8ce8-4e1c17c4c1e4\") " pod="openshift-insights/insights-operator-555d9d7885-jv9pk" Jul 17 07:12:16.947268 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.947166 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7hmb\" (UniqueName: \"kubernetes.io/projected/2ce00a54-15e9-42b7-b3a8-9e58d1101558-kube-api-access-b7hmb\") pod \"cluster-monitoring-operator-76cf8966f8-xr6pg\" (UID: \"2ce00a54-15e9-42b7-b3a8-9e58d1101558\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xr6pg" Jul 17 07:12:16.947698 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.947677 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/27538d1e-992b-4651-a2d2-7a7ecf442806-trusted-ca\") pod \"console-operator-565b7b569d-xpxv5\" (UID: \"27538d1e-992b-4651-a2d2-7a7ecf442806\") " pod="openshift-console-operator/console-operator-565b7b569d-xpxv5" Jul 17 07:12:16.947752 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.947686 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/27538d1e-992b-4651-a2d2-7a7ecf442806-config\") pod \"console-operator-565b7b569d-xpxv5\" (UID: \"27538d1e-992b-4651-a2d2-7a7ecf442806\") " pod="openshift-console-operator/console-operator-565b7b569d-xpxv5" Jul 17 07:12:16.949873 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.949848 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/27538d1e-992b-4651-a2d2-7a7ecf442806-serving-cert\") pod \"console-operator-565b7b569d-xpxv5\" (UID: \"27538d1e-992b-4651-a2d2-7a7ecf442806\") " pod="openshift-console-operator/console-operator-565b7b569d-xpxv5" Jul 17 07:12:16.949960 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.949892 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/254d445f-a7c6-49e6-a9a6-51b738ccdab2-default-certificate\") pod \"router-default-6784546c7b-l47vf\" (UID: \"254d445f-a7c6-49e6-a9a6-51b738ccdab2\") " pod="openshift-ingress/router-default-6784546c7b-l47vf" Jul 17 07:12:16.950025 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.949960 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/254d445f-a7c6-49e6-a9a6-51b738ccdab2-stats-auth\") pod \"router-default-6784546c7b-l47vf\" (UID: \"254d445f-a7c6-49e6-a9a6-51b738ccdab2\") " pod="openshift-ingress/router-default-6784546c7b-l47vf" Jul 17 07:12:16.956850 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.956831 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cckw7\" (UniqueName: \"kubernetes.io/projected/2b81b5e5-a313-4e9c-87e6-4b0242f46a78-kube-api-access-cckw7\") pod \"cluster-samples-operator-79b7c55f87-z6kdb\" (UID: \"2b81b5e5-a313-4e9c-87e6-4b0242f46a78\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-z6kdb" Jul 17 07:12:16.956850 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.956842 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rk7b\" (UniqueName: \"kubernetes.io/projected/27538d1e-992b-4651-a2d2-7a7ecf442806-kube-api-access-2rk7b\") pod \"console-operator-565b7b569d-xpxv5\" (UID: \"27538d1e-992b-4651-a2d2-7a7ecf442806\") " pod="openshift-console-operator/console-operator-565b7b569d-xpxv5" Jul 17 07:12:16.957373 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.957347 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrqmd\" (UniqueName: \"kubernetes.io/projected/254d445f-a7c6-49e6-a9a6-51b738ccdab2-kube-api-access-lrqmd\") pod \"router-default-6784546c7b-l47vf\" (UID: \"254d445f-a7c6-49e6-a9a6-51b738ccdab2\") " pod="openshift-ingress/router-default-6784546c7b-l47vf" Jul 17 07:12:16.990067 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:16.990049 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-xpxv5" Jul 17 07:12:17.048401 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:17.048376 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2ce00a54-15e9-42b7-b3a8-9e58d1101558-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-xr6pg\" (UID: \"2ce00a54-15e9-42b7-b3a8-9e58d1101558\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xr6pg" Jul 17 07:12:17.048515 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:17.048419 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ce00a54-15e9-42b7-b3a8-9e58d1101558-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-xr6pg\" (UID: \"2ce00a54-15e9-42b7-b3a8-9e58d1101558\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xr6pg" Jul 17 07:12:17.048515 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:17.048453 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/2a3b3827-cd5d-4292-8ce8-4e1c17c4c1e4-snapshots\") pod \"insights-operator-555d9d7885-jv9pk\" (UID: \"2a3b3827-cd5d-4292-8ce8-4e1c17c4c1e4\") " pod="openshift-insights/insights-operator-555d9d7885-jv9pk" Jul 17 07:12:17.048627 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:17.048519 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2a3b3827-cd5d-4292-8ce8-4e1c17c4c1e4-serving-cert\") pod \"insights-operator-555d9d7885-jv9pk\" (UID: \"2a3b3827-cd5d-4292-8ce8-4e1c17c4c1e4\") " pod="openshift-insights/insights-operator-555d9d7885-jv9pk" Jul 17 07:12:17.048627 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:17.048547 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7hmb\" (UniqueName: \"kubernetes.io/projected/2ce00a54-15e9-42b7-b3a8-9e58d1101558-kube-api-access-b7hmb\") pod \"cluster-monitoring-operator-76cf8966f8-xr6pg\" (UID: \"2ce00a54-15e9-42b7-b3a8-9e58d1101558\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xr6pg" Jul 17 07:12:17.048627 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:17.048563 2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 17 07:12:17.048769 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:17.048627 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a3b3827-cd5d-4292-8ce8-4e1c17c4c1e4-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-jv9pk\" (UID: \"2a3b3827-cd5d-4292-8ce8-4e1c17c4c1e4\") " pod="openshift-insights/insights-operator-555d9d7885-jv9pk" Jul 17 07:12:17.048769 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:17.048653 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ce00a54-15e9-42b7-b3a8-9e58d1101558-cluster-monitoring-operator-tls podName:2ce00a54-15e9-42b7-b3a8-9e58d1101558 nodeName:}" failed. No retries permitted until 2026-07-17 07:12:17.548628877 +0000 UTC m=+42.033317834 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2ce00a54-15e9-42b7-b3a8-9e58d1101558-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-xr6pg" (UID: "2ce00a54-15e9-42b7-b3a8-9e58d1101558") : secret "cluster-monitoring-operator-tls" not found Jul 17 07:12:17.048769 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:17.048696 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a3b3827-cd5d-4292-8ce8-4e1c17c4c1e4-service-ca-bundle\") pod \"insights-operator-555d9d7885-jv9pk\" (UID: \"2a3b3827-cd5d-4292-8ce8-4e1c17c4c1e4\") " pod="openshift-insights/insights-operator-555d9d7885-jv9pk" Jul 17 07:12:17.048769 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:17.048728 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k72rq\" (UniqueName: \"kubernetes.io/projected/2a3b3827-cd5d-4292-8ce8-4e1c17c4c1e4-kube-api-access-k72rq\") pod \"insights-operator-555d9d7885-jv9pk\" (UID: \"2a3b3827-cd5d-4292-8ce8-4e1c17c4c1e4\") " pod="openshift-insights/insights-operator-555d9d7885-jv9pk" Jul 17 07:12:17.048965 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:17.048783 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57csl\" (UniqueName: \"kubernetes.io/projected/6462d06e-f55f-44cc-9913-ef8120a6f4cc-kube-api-access-57csl\") pod \"volume-data-source-validator-78cf4c6f6-xgtz8\" (UID: \"6462d06e-f55f-44cc-9913-ef8120a6f4cc\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-xgtz8" Jul 17 07:12:17.048965 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:17.048833 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2a3b3827-cd5d-4292-8ce8-4e1c17c4c1e4-tmp\") pod \"insights-operator-555d9d7885-jv9pk\" (UID: \"2a3b3827-cd5d-4292-8ce8-4e1c17c4c1e4\") " pod="openshift-insights/insights-operator-555d9d7885-jv9pk" Jul 17 07:12:17.049116 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:17.049093 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/2a3b3827-cd5d-4292-8ce8-4e1c17c4c1e4-snapshots\") pod \"insights-operator-555d9d7885-jv9pk\" (UID: \"2a3b3827-cd5d-4292-8ce8-4e1c17c4c1e4\") " pod="openshift-insights/insights-operator-555d9d7885-jv9pk" Jul 17 07:12:17.049188 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:17.049169 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2a3b3827-cd5d-4292-8ce8-4e1c17c4c1e4-tmp\") pod \"insights-operator-555d9d7885-jv9pk\" (UID: \"2a3b3827-cd5d-4292-8ce8-4e1c17c4c1e4\") " pod="openshift-insights/insights-operator-555d9d7885-jv9pk" Jul 17 07:12:17.049244 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:17.049202 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2ce00a54-15e9-42b7-b3a8-9e58d1101558-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-xr6pg\" (UID: \"2ce00a54-15e9-42b7-b3a8-9e58d1101558\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xr6pg" Jul 17 07:12:17.049581 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:17.049563 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a3b3827-cd5d-4292-8ce8-4e1c17c4c1e4-service-ca-bundle\") pod \"insights-operator-555d9d7885-jv9pk\" (UID: \"2a3b3827-cd5d-4292-8ce8-4e1c17c4c1e4\") " pod="openshift-insights/insights-operator-555d9d7885-jv9pk" Jul 17 07:12:17.049646 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:17.049619 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a3b3827-cd5d-4292-8ce8-4e1c17c4c1e4-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-jv9pk\" (UID: \"2a3b3827-cd5d-4292-8ce8-4e1c17c4c1e4\") " pod="openshift-insights/insights-operator-555d9d7885-jv9pk" Jul 17 07:12:17.050832 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:17.050810 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2a3b3827-cd5d-4292-8ce8-4e1c17c4c1e4-serving-cert\") pod \"insights-operator-555d9d7885-jv9pk\" (UID: \"2a3b3827-cd5d-4292-8ce8-4e1c17c4c1e4\") " pod="openshift-insights/insights-operator-555d9d7885-jv9pk" Jul 17 07:12:17.058204 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:17.058185 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k72rq\" (UniqueName: \"kubernetes.io/projected/2a3b3827-cd5d-4292-8ce8-4e1c17c4c1e4-kube-api-access-k72rq\") pod \"insights-operator-555d9d7885-jv9pk\" (UID: \"2a3b3827-cd5d-4292-8ce8-4e1c17c4c1e4\") " pod="openshift-insights/insights-operator-555d9d7885-jv9pk" Jul 17 07:12:17.058305 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:17.058185 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7hmb\" (UniqueName: \"kubernetes.io/projected/2ce00a54-15e9-42b7-b3a8-9e58d1101558-kube-api-access-b7hmb\") pod \"cluster-monitoring-operator-76cf8966f8-xr6pg\" (UID: \"2ce00a54-15e9-42b7-b3a8-9e58d1101558\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xr6pg" Jul 17 07:12:17.058459 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:17.058437 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-57csl\" (UniqueName: \"kubernetes.io/projected/6462d06e-f55f-44cc-9913-ef8120a6f4cc-kube-api-access-57csl\") pod \"volume-data-source-validator-78cf4c6f6-xgtz8\" (UID: \"6462d06e-f55f-44cc-9913-ef8120a6f4cc\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-xgtz8" Jul 17 07:12:17.091174 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:17.091145 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-xgtz8" Jul 17 07:12:17.135639 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:17.135611 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-jv9pk" Jul 17 07:12:17.151492 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:17.151467 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-xpxv5"] Jul 17 07:12:17.154013 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:12:17.153987 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27538d1e_992b_4651_a2d2_7a7ecf442806.slice/crio-6197130cdc6c05788307300ea8d246616dd37b15c800ca57b7cf9c031a7b1b2e WatchSource:0}: Error finding container 6197130cdc6c05788307300ea8d246616dd37b15c800ca57b7cf9c031a7b1b2e: Status 404 returned error can't find the container with id 6197130cdc6c05788307300ea8d246616dd37b15c800ca57b7cf9c031a7b1b2e Jul 17 07:12:17.234909 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:17.234868 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-xgtz8"] Jul 17 07:12:17.238617 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:12:17.238583 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6462d06e_f55f_44cc_9913_ef8120a6f4cc.slice/crio-8b66c62cd4d5961a28e71e71e7ccc40fecc59eed8576cca3c181cb525db887cd WatchSource:0}: Error finding container 8b66c62cd4d5961a28e71e71e7ccc40fecc59eed8576cca3c181cb525db887cd: Status 404 returned error can't find the container with id 8b66c62cd4d5961a28e71e71e7ccc40fecc59eed8576cca3c181cb525db887cd Jul 17 07:12:17.303050 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:17.303022 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-xgtz8" event={"ID":"6462d06e-f55f-44cc-9913-ef8120a6f4cc","Type":"ContainerStarted","Data":"8b66c62cd4d5961a28e71e71e7ccc40fecc59eed8576cca3c181cb525db887cd"} Jul 17 07:12:17.304165 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:17.304141 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-xpxv5" event={"ID":"27538d1e-992b-4651-a2d2-7a7ecf442806","Type":"ContainerStarted","Data":"6197130cdc6c05788307300ea8d246616dd37b15c800ca57b7cf9c031a7b1b2e"} Jul 17 07:12:17.308665 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:17.308640 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-jv9pk"] Jul 17 07:12:17.312898 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:12:17.312873 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a3b3827_cd5d_4292_8ce8_4e1c17c4c1e4.slice/crio-71e18356080354d2ddb158a9e19e13ab603370808598783997ae45ddd54ab645 WatchSource:0}: Error finding container 71e18356080354d2ddb158a9e19e13ab603370808598783997ae45ddd54ab645: Status 404 returned error can't find the container with id 71e18356080354d2ddb158a9e19e13ab603370808598783997ae45ddd54ab645 Jul 17 07:12:17.452509 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:17.452428 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/254d445f-a7c6-49e6-a9a6-51b738ccdab2-metrics-certs\") pod \"router-default-6784546c7b-l47vf\" (UID: \"254d445f-a7c6-49e6-a9a6-51b738ccdab2\") " pod="openshift-ingress/router-default-6784546c7b-l47vf" Jul 17 07:12:17.452665 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:17.452541 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2b81b5e5-a313-4e9c-87e6-4b0242f46a78-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-z6kdb\" (UID: \"2b81b5e5-a313-4e9c-87e6-4b0242f46a78\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-z6kdb" Jul 17 07:12:17.452665 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:17.452576 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/254d445f-a7c6-49e6-a9a6-51b738ccdab2-service-ca-bundle\") pod \"router-default-6784546c7b-l47vf\" (UID: \"254d445f-a7c6-49e6-a9a6-51b738ccdab2\") " pod="openshift-ingress/router-default-6784546c7b-l47vf" Jul 17 07:12:17.452665 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:17.452590 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 17 07:12:17.452665 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:17.452655 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/254d445f-a7c6-49e6-a9a6-51b738ccdab2-metrics-certs podName:254d445f-a7c6-49e6-a9a6-51b738ccdab2 nodeName:}" failed. No retries permitted until 2026-07-17 07:12:18.452633119 +0000 UTC m=+42.937322074 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/254d445f-a7c6-49e6-a9a6-51b738ccdab2-metrics-certs") pod "router-default-6784546c7b-l47vf" (UID: "254d445f-a7c6-49e6-a9a6-51b738ccdab2") : secret "router-metrics-certs-default" not found Jul 17 07:12:17.452862 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:17.452711 2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 17 07:12:17.452862 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:17.452714 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/254d445f-a7c6-49e6-a9a6-51b738ccdab2-service-ca-bundle podName:254d445f-a7c6-49e6-a9a6-51b738ccdab2 nodeName:}" failed. No retries permitted until 2026-07-17 07:12:18.452698411 +0000 UTC m=+42.937387363 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/254d445f-a7c6-49e6-a9a6-51b738ccdab2-service-ca-bundle") pod "router-default-6784546c7b-l47vf" (UID: "254d445f-a7c6-49e6-a9a6-51b738ccdab2") : configmap references non-existent config key: service-ca.crt Jul 17 07:12:17.452862 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:17.452793 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b81b5e5-a313-4e9c-87e6-4b0242f46a78-samples-operator-tls podName:2b81b5e5-a313-4e9c-87e6-4b0242f46a78 nodeName:}" failed. No retries permitted until 2026-07-17 07:12:18.452781932 +0000 UTC m=+42.937470888 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2b81b5e5-a313-4e9c-87e6-4b0242f46a78-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-z6kdb" (UID: "2b81b5e5-a313-4e9c-87e6-4b0242f46a78") : secret "samples-operator-tls" not found Jul 17 07:12:17.553829 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:17.553800 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ce00a54-15e9-42b7-b3a8-9e58d1101558-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-xr6pg\" (UID: \"2ce00a54-15e9-42b7-b3a8-9e58d1101558\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xr6pg" Jul 17 07:12:17.553946 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:17.553897 2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 17 07:12:17.553946 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:17.553940 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ce00a54-15e9-42b7-b3a8-9e58d1101558-cluster-monitoring-operator-tls podName:2ce00a54-15e9-42b7-b3a8-9e58d1101558 nodeName:}" failed. No retries permitted until 2026-07-17 07:12:18.553927067 +0000 UTC m=+43.038616021 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2ce00a54-15e9-42b7-b3a8-9e58d1101558-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-xr6pg" (UID: "2ce00a54-15e9-42b7-b3a8-9e58d1101558") : secret "cluster-monitoring-operator-tls" not found Jul 17 07:12:17.659751 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:17.659712 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-44ftt"] Jul 17 07:12:17.697344 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:17.697315 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-44ftt"] Jul 17 07:12:17.697469 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:17.697368 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-44ftt" Jul 17 07:12:17.700062 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:17.700039 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 17 07:12:17.700206 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:17.700064 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-p5znm\"" Jul 17 07:12:17.700206 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:17.700126 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 17 07:12:17.700314 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:17.700272 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 17 07:12:17.700408 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:17.700390 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 17 07:12:17.856522 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:17.856480 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4c8e8c69-d8f8-4285-bed5-b527c4d915f8-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-44ftt\" (UID: \"4c8e8c69-d8f8-4285-bed5-b527c4d915f8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-44ftt" Jul 17 07:12:17.856714 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:17.856631 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4c8e8c69-d8f8-4285-bed5-b527c4d915f8-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-44ftt\" (UID: \"4c8e8c69-d8f8-4285-bed5-b527c4d915f8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-44ftt" Jul 17 07:12:17.856795 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:17.856778 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whr69\" (UniqueName: \"kubernetes.io/projected/4c8e8c69-d8f8-4285-bed5-b527c4d915f8-kube-api-access-whr69\") pod \"kube-storage-version-migrator-operator-7658d9b749-44ftt\" (UID: \"4c8e8c69-d8f8-4285-bed5-b527c4d915f8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-44ftt" Jul 17 07:12:17.957715 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:17.957629 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whr69\" (UniqueName: \"kubernetes.io/projected/4c8e8c69-d8f8-4285-bed5-b527c4d915f8-kube-api-access-whr69\") pod \"kube-storage-version-migrator-operator-7658d9b749-44ftt\" (UID: \"4c8e8c69-d8f8-4285-bed5-b527c4d915f8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-44ftt" Jul 17 07:12:17.957856 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:17.957728 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4c8e8c69-d8f8-4285-bed5-b527c4d915f8-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-44ftt\" (UID: \"4c8e8c69-d8f8-4285-bed5-b527c4d915f8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-44ftt" Jul 17 07:12:17.957856 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:17.957837 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4c8e8c69-d8f8-4285-bed5-b527c4d915f8-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-44ftt\" (UID: \"4c8e8c69-d8f8-4285-bed5-b527c4d915f8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-44ftt" Jul 17 07:12:17.958405 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:17.958383 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4c8e8c69-d8f8-4285-bed5-b527c4d915f8-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-44ftt\" (UID: \"4c8e8c69-d8f8-4285-bed5-b527c4d915f8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-44ftt" Jul 17 07:12:17.960213 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:17.960193 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4c8e8c69-d8f8-4285-bed5-b527c4d915f8-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-44ftt\" (UID: \"4c8e8c69-d8f8-4285-bed5-b527c4d915f8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-44ftt" Jul 17 07:12:17.968802 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:17.968778 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whr69\" (UniqueName: \"kubernetes.io/projected/4c8e8c69-d8f8-4285-bed5-b527c4d915f8-kube-api-access-whr69\") pod \"kube-storage-version-migrator-operator-7658d9b749-44ftt\" (UID: \"4c8e8c69-d8f8-4285-bed5-b527c4d915f8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-44ftt" Jul 17 07:12:18.005783 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:18.005761 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-44ftt" Jul 17 07:12:18.177156 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:18.177122 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-44ftt"] Jul 17 07:12:18.181512 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:12:18.181465 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c8e8c69_d8f8_4285_bed5_b527c4d915f8.slice/crio-71b8daa9c954c5e421f1ab7d6506838bbc08a92136869f876416fd122e781871 WatchSource:0}: Error finding container 71b8daa9c954c5e421f1ab7d6506838bbc08a92136869f876416fd122e781871: Status 404 returned error can't find the container with id 71b8daa9c954c5e421f1ab7d6506838bbc08a92136869f876416fd122e781871 Jul 17 07:12:18.308789 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:18.308671 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-44ftt" event={"ID":"4c8e8c69-d8f8-4285-bed5-b527c4d915f8","Type":"ContainerStarted","Data":"71b8daa9c954c5e421f1ab7d6506838bbc08a92136869f876416fd122e781871"} Jul 17 07:12:18.310349 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:18.310319 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-jv9pk" event={"ID":"2a3b3827-cd5d-4292-8ce8-4e1c17c4c1e4","Type":"ContainerStarted","Data":"71e18356080354d2ddb158a9e19e13ab603370808598783997ae45ddd54ab645"} Jul 17 07:12:18.463846 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:18.463810 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/254d445f-a7c6-49e6-a9a6-51b738ccdab2-metrics-certs\") pod \"router-default-6784546c7b-l47vf\" (UID: \"254d445f-a7c6-49e6-a9a6-51b738ccdab2\") " pod="openshift-ingress/router-default-6784546c7b-l47vf" Jul 17 07:12:18.464017 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:18.463900 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2b81b5e5-a313-4e9c-87e6-4b0242f46a78-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-z6kdb\" (UID: \"2b81b5e5-a313-4e9c-87e6-4b0242f46a78\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-z6kdb" Jul 17 07:12:18.464017 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:18.463933 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/254d445f-a7c6-49e6-a9a6-51b738ccdab2-service-ca-bundle\") pod \"router-default-6784546c7b-l47vf\" (UID: \"254d445f-a7c6-49e6-a9a6-51b738ccdab2\") " pod="openshift-ingress/router-default-6784546c7b-l47vf" Jul 17 07:12:18.464017 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:18.463971 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 17 07:12:18.464188 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:18.464037 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/254d445f-a7c6-49e6-a9a6-51b738ccdab2-metrics-certs podName:254d445f-a7c6-49e6-a9a6-51b738ccdab2 nodeName:}" failed. No retries permitted until 2026-07-17 07:12:20.464018902 +0000 UTC m=+44.948707866 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/254d445f-a7c6-49e6-a9a6-51b738ccdab2-metrics-certs") pod "router-default-6784546c7b-l47vf" (UID: "254d445f-a7c6-49e6-a9a6-51b738ccdab2") : secret "router-metrics-certs-default" not found Jul 17 07:12:18.464188 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:18.464055 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/254d445f-a7c6-49e6-a9a6-51b738ccdab2-service-ca-bundle podName:254d445f-a7c6-49e6-a9a6-51b738ccdab2 nodeName:}" failed. No retries permitted until 2026-07-17 07:12:20.464047335 +0000 UTC m=+44.948736287 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/254d445f-a7c6-49e6-a9a6-51b738ccdab2-service-ca-bundle") pod "router-default-6784546c7b-l47vf" (UID: "254d445f-a7c6-49e6-a9a6-51b738ccdab2") : configmap references non-existent config key: service-ca.crt Jul 17 07:12:18.464188 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:18.464075 2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 17 07:12:18.464188 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:18.464150 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b81b5e5-a313-4e9c-87e6-4b0242f46a78-samples-operator-tls podName:2b81b5e5-a313-4e9c-87e6-4b0242f46a78 nodeName:}" failed. No retries permitted until 2026-07-17 07:12:20.464130154 +0000 UTC m=+44.948819110 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2b81b5e5-a313-4e9c-87e6-4b0242f46a78-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-z6kdb" (UID: "2b81b5e5-a313-4e9c-87e6-4b0242f46a78") : secret "samples-operator-tls" not found Jul 17 07:12:18.564667 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:18.564533 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ce00a54-15e9-42b7-b3a8-9e58d1101558-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-xr6pg\" (UID: \"2ce00a54-15e9-42b7-b3a8-9e58d1101558\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xr6pg" Jul 17 07:12:18.564836 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:18.564795 2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 17 07:12:18.564897 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:18.564876 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ce00a54-15e9-42b7-b3a8-9e58d1101558-cluster-monitoring-operator-tls podName:2ce00a54-15e9-42b7-b3a8-9e58d1101558 nodeName:}" failed. No retries permitted until 2026-07-17 07:12:20.564856138 +0000 UTC m=+45.049545095 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2ce00a54-15e9-42b7-b3a8-9e58d1101558-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-xr6pg" (UID: "2ce00a54-15e9-42b7-b3a8-9e58d1101558") : secret "cluster-monitoring-operator-tls" not found Jul 17 07:12:19.069524 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:19.069491 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0161fccb-b49c-4592-8a96-b79d237fe749-registry-tls\") pod \"image-registry-6675666ffd-gnf79\" (UID: \"0161fccb-b49c-4592-8a96-b79d237fe749\") " pod="openshift-image-registry/image-registry-6675666ffd-gnf79" Jul 17 07:12:19.069720 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:19.069621 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b1d8b7b7-173b-4e59-acce-7fbe2b1f5465-metrics-tls\") pod \"dns-default-vc255\" (UID: \"b1d8b7b7-173b-4e59-acce-7fbe2b1f5465\") " pod="openshift-dns/dns-default-vc255" Jul 17 07:12:19.069720 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:19.069645 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 17 07:12:19.069720 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:19.069677 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6675666ffd-gnf79: secret "image-registry-tls" not found Jul 17 07:12:19.069877 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:19.069734 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 17 07:12:19.069877 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:19.069740 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0161fccb-b49c-4592-8a96-b79d237fe749-registry-tls podName:0161fccb-b49c-4592-8a96-b79d237fe749 nodeName:}" failed. No retries permitted until 2026-07-17 07:12:27.069718814 +0000 UTC m=+51.554407770 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0161fccb-b49c-4592-8a96-b79d237fe749-registry-tls") pod "image-registry-6675666ffd-gnf79" (UID: "0161fccb-b49c-4592-8a96-b79d237fe749") : secret "image-registry-tls" not found Jul 17 07:12:19.069877 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:19.069809 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1d8b7b7-173b-4e59-acce-7fbe2b1f5465-metrics-tls podName:b1d8b7b7-173b-4e59-acce-7fbe2b1f5465 nodeName:}" failed. No retries permitted until 2026-07-17 07:12:27.069788121 +0000 UTC m=+51.554477078 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b1d8b7b7-173b-4e59-acce-7fbe2b1f5465-metrics-tls") pod "dns-default-vc255" (UID: "b1d8b7b7-173b-4e59-acce-7fbe2b1f5465") : secret "dns-default-metrics-tls" not found Jul 17 07:12:20.483384 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:20.483348 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/254d445f-a7c6-49e6-a9a6-51b738ccdab2-metrics-certs\") pod \"router-default-6784546c7b-l47vf\" (UID: \"254d445f-a7c6-49e6-a9a6-51b738ccdab2\") " pod="openshift-ingress/router-default-6784546c7b-l47vf" Jul 17 07:12:20.483766 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:20.483431 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2b81b5e5-a313-4e9c-87e6-4b0242f46a78-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-z6kdb\" (UID: \"2b81b5e5-a313-4e9c-87e6-4b0242f46a78\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-z6kdb" Jul 17 07:12:20.483766 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:20.483452 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/254d445f-a7c6-49e6-a9a6-51b738ccdab2-service-ca-bundle\") pod \"router-default-6784546c7b-l47vf\" (UID: \"254d445f-a7c6-49e6-a9a6-51b738ccdab2\") " pod="openshift-ingress/router-default-6784546c7b-l47vf" Jul 17 07:12:20.483766 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:20.483511 2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 17 07:12:20.483766 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:20.483512 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 17 07:12:20.483766 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:20.483562 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/254d445f-a7c6-49e6-a9a6-51b738ccdab2-service-ca-bundle podName:254d445f-a7c6-49e6-a9a6-51b738ccdab2 nodeName:}" failed. No retries permitted until 2026-07-17 07:12:24.483543147 +0000 UTC m=+48.968232102 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/254d445f-a7c6-49e6-a9a6-51b738ccdab2-service-ca-bundle") pod "router-default-6784546c7b-l47vf" (UID: "254d445f-a7c6-49e6-a9a6-51b738ccdab2") : configmap references non-existent config key: service-ca.crt Jul 17 07:12:20.483766 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:20.483586 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b81b5e5-a313-4e9c-87e6-4b0242f46a78-samples-operator-tls podName:2b81b5e5-a313-4e9c-87e6-4b0242f46a78 nodeName:}" failed. No retries permitted until 2026-07-17 07:12:24.483577212 +0000 UTC m=+48.968266174 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2b81b5e5-a313-4e9c-87e6-4b0242f46a78-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-z6kdb" (UID: "2b81b5e5-a313-4e9c-87e6-4b0242f46a78") : secret "samples-operator-tls" not found Jul 17 07:12:20.483766 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:20.483599 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/254d445f-a7c6-49e6-a9a6-51b738ccdab2-metrics-certs podName:254d445f-a7c6-49e6-a9a6-51b738ccdab2 nodeName:}" failed. No retries permitted until 2026-07-17 07:12:24.483592402 +0000 UTC m=+48.968281363 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/254d445f-a7c6-49e6-a9a6-51b738ccdab2-metrics-certs") pod "router-default-6784546c7b-l47vf" (UID: "254d445f-a7c6-49e6-a9a6-51b738ccdab2") : secret "router-metrics-certs-default" not found Jul 17 07:12:20.584913 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:20.584874 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ce00a54-15e9-42b7-b3a8-9e58d1101558-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-xr6pg\" (UID: \"2ce00a54-15e9-42b7-b3a8-9e58d1101558\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xr6pg" Jul 17 07:12:20.585051 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:20.585022 2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 17 07:12:20.585106 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:20.585087 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ce00a54-15e9-42b7-b3a8-9e58d1101558-cluster-monitoring-operator-tls podName:2ce00a54-15e9-42b7-b3a8-9e58d1101558 nodeName:}" failed. No retries permitted until 2026-07-17 07:12:24.585070248 +0000 UTC m=+49.069759203 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2ce00a54-15e9-42b7-b3a8-9e58d1101558-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-xr6pg" (UID: "2ce00a54-15e9-42b7-b3a8-9e58d1101558") : secret "cluster-monitoring-operator-tls" not found Jul 17 07:12:22.323912 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:22.323872 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-44ftt" event={"ID":"4c8e8c69-d8f8-4285-bed5-b527c4d915f8","Type":"ContainerStarted","Data":"d2e6ed6a3081c66fb2d3000f2f6d97a4f607c165a5d353bd2013347999d45d0b"} Jul 17 07:12:22.325498 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:22.325474 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-jv9pk" event={"ID":"2a3b3827-cd5d-4292-8ce8-4e1c17c4c1e4","Type":"ContainerStarted","Data":"6175b80a3b217fe5e6cf6e78ad1e774a5ab555af362b8a92002a8b0c098c03ce"} Jul 17 07:12:22.326873 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:22.326853 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-xgtz8" event={"ID":"6462d06e-f55f-44cc-9913-ef8120a6f4cc","Type":"ContainerStarted","Data":"8f6dd380af1c8679beba7a017868594ad755189717caaa377ac8115623fc0f1f"} Jul 17 07:12:22.328480 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:22.328460 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-xpxv5_27538d1e-992b-4651-a2d2-7a7ecf442806/console-operator/0.log" Jul 17 07:12:22.328583 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:22.328492 2568 generic.go:356] "Generic (PLEG): container finished" podID="27538d1e-992b-4651-a2d2-7a7ecf442806" containerID="547a3aab6f47d694148ce95f82be7838875604325392113fe0c28c84f85dea4d" exitCode=255 Jul 17 07:12:22.328583 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:22.328519 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-xpxv5" event={"ID":"27538d1e-992b-4651-a2d2-7a7ecf442806","Type":"ContainerDied","Data":"547a3aab6f47d694148ce95f82be7838875604325392113fe0c28c84f85dea4d"} Jul 17 07:12:22.328710 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:22.328695 2568 scope.go:117] "RemoveContainer" containerID="547a3aab6f47d694148ce95f82be7838875604325392113fe0c28c84f85dea4d" Jul 17 07:12:22.340654 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:22.340612 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-44ftt" podStartSLOduration=1.7593955879999998 podStartE2EDuration="5.340595503s" totalImagesPullingTime="3.581199915s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 07:12:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:12:22.339422978 +0000 UTC m=+46.824111958" watchObservedRunningTime="2026-07-17 07:12:22.340595503 +0000 UTC m=+46.825284479" Jul 17 07:12:22.354144 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:22.354096 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-xgtz8" podStartSLOduration=2.136047803 podStartE2EDuration="6.354078752s" totalImagesPullingTime="4.218030949s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 07:12:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:12:22.353636331 +0000 UTC m=+46.838325306" watchObservedRunningTime="2026-07-17 07:12:22.354078752 +0000 UTC m=+46.838767727" Jul 17 07:12:22.393936 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:22.393889 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-jv9pk" podStartSLOduration=2.243342989 podStartE2EDuration="6.393873614s" totalImagesPullingTime="4.150530625s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 07:12:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:12:22.393607158 +0000 UTC m=+46.878296132" watchObservedRunningTime="2026-07-17 07:12:22.393873614 +0000 UTC m=+46.878562602" Jul 17 07:12:23.000634 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:23.000596 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-khqt9"] Jul 17 07:12:23.022862 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:23.022835 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-khqt9"] Jul 17 07:12:23.023010 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:23.022948 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-khqt9" Jul 17 07:12:23.027906 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:23.027883 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 17 07:12:23.029225 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:23.029206 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 17 07:12:23.029225 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:23.029216 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-gcxnf\"" Jul 17 07:12:23.208429 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:23.208389 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjlfs\" (UniqueName: \"kubernetes.io/projected/9a4455d4-a377-4848-a454-54f3710b4a02-kube-api-access-xjlfs\") pod \"insights-runtime-extractor-khqt9\" (UID: \"9a4455d4-a377-4848-a454-54f3710b4a02\") " pod="openshift-insights/insights-runtime-extractor-khqt9" Jul 17 07:12:23.208609 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:23.208476 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9a4455d4-a377-4848-a454-54f3710b4a02-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-khqt9\" (UID: \"9a4455d4-a377-4848-a454-54f3710b4a02\") " pod="openshift-insights/insights-runtime-extractor-khqt9" Jul 17 07:12:23.208609 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:23.208505 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9a4455d4-a377-4848-a454-54f3710b4a02-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-khqt9\" (UID: \"9a4455d4-a377-4848-a454-54f3710b4a02\") " pod="openshift-insights/insights-runtime-extractor-khqt9" Jul 17 07:12:23.208609 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:23.208552 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9a4455d4-a377-4848-a454-54f3710b4a02-data-volume\") pod \"insights-runtime-extractor-khqt9\" (UID: \"9a4455d4-a377-4848-a454-54f3710b4a02\") " pod="openshift-insights/insights-runtime-extractor-khqt9" Jul 17 07:12:23.208734 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:23.208710 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9a4455d4-a377-4848-a454-54f3710b4a02-crio-socket\") pod \"insights-runtime-extractor-khqt9\" (UID: \"9a4455d4-a377-4848-a454-54f3710b4a02\") " pod="openshift-insights/insights-runtime-extractor-khqt9" Jul 17 07:12:23.309451 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:23.309378 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjlfs\" (UniqueName: \"kubernetes.io/projected/9a4455d4-a377-4848-a454-54f3710b4a02-kube-api-access-xjlfs\") pod \"insights-runtime-extractor-khqt9\" (UID: \"9a4455d4-a377-4848-a454-54f3710b4a02\") " pod="openshift-insights/insights-runtime-extractor-khqt9" Jul 17 07:12:23.309451 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:23.309435 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9a4455d4-a377-4848-a454-54f3710b4a02-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-khqt9\" (UID: \"9a4455d4-a377-4848-a454-54f3710b4a02\") " pod="openshift-insights/insights-runtime-extractor-khqt9" Jul 17 07:12:23.309638 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:23.309458 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9a4455d4-a377-4848-a454-54f3710b4a02-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-khqt9\" (UID: \"9a4455d4-a377-4848-a454-54f3710b4a02\") " pod="openshift-insights/insights-runtime-extractor-khqt9" Jul 17 07:12:23.309638 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:23.309488 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9a4455d4-a377-4848-a454-54f3710b4a02-data-volume\") pod \"insights-runtime-extractor-khqt9\" (UID: \"9a4455d4-a377-4848-a454-54f3710b4a02\") " pod="openshift-insights/insights-runtime-extractor-khqt9" Jul 17 07:12:23.309638 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:23.309533 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 17 07:12:23.309638 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:23.309598 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9a4455d4-a377-4848-a454-54f3710b4a02-kube-rbac-proxy-cm podName:9a4455d4-a377-4848-a454-54f3710b4a02 nodeName:}" failed. No retries permitted until 2026-07-17 07:12:23.809578161 +0000 UTC m=+48.294267130 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9a4455d4-a377-4848-a454-54f3710b4a02-kube-rbac-proxy-cm") pod "insights-runtime-extractor-khqt9" (UID: "9a4455d4-a377-4848-a454-54f3710b4a02") : configmap "kube-rbac-proxy" not found Jul 17 07:12:23.309638 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:23.309613 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 17 07:12:23.309889 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:23.309650 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9a4455d4-a377-4848-a454-54f3710b4a02-crio-socket\") pod \"insights-runtime-extractor-khqt9\" (UID: \"9a4455d4-a377-4848-a454-54f3710b4a02\") " pod="openshift-insights/insights-runtime-extractor-khqt9" Jul 17 07:12:23.309889 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:23.309690 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a4455d4-a377-4848-a454-54f3710b4a02-insights-runtime-extractor-tls podName:9a4455d4-a377-4848-a454-54f3710b4a02 nodeName:}" failed. No retries permitted until 2026-07-17 07:12:23.809673848 +0000 UTC m=+48.294362805 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9a4455d4-a377-4848-a454-54f3710b4a02-insights-runtime-extractor-tls") pod "insights-runtime-extractor-khqt9" (UID: "9a4455d4-a377-4848-a454-54f3710b4a02") : secret "insights-runtime-extractor-tls" not found Jul 17 07:12:23.309889 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:23.309848 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9a4455d4-a377-4848-a454-54f3710b4a02-crio-socket\") pod \"insights-runtime-extractor-khqt9\" (UID: \"9a4455d4-a377-4848-a454-54f3710b4a02\") " pod="openshift-insights/insights-runtime-extractor-khqt9" Jul 17 07:12:23.320098 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:23.320081 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9a4455d4-a377-4848-a454-54f3710b4a02-data-volume\") pod \"insights-runtime-extractor-khqt9\" (UID: \"9a4455d4-a377-4848-a454-54f3710b4a02\") " pod="openshift-insights/insights-runtime-extractor-khqt9" Jul 17 07:12:23.325993 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:23.325972 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjlfs\" (UniqueName: \"kubernetes.io/projected/9a4455d4-a377-4848-a454-54f3710b4a02-kube-api-access-xjlfs\") pod \"insights-runtime-extractor-khqt9\" (UID: \"9a4455d4-a377-4848-a454-54f3710b4a02\") " pod="openshift-insights/insights-runtime-extractor-khqt9" Jul 17 07:12:23.333420 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:23.333400 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-xpxv5_27538d1e-992b-4651-a2d2-7a7ecf442806/console-operator/1.log" Jul 17 07:12:23.333866 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:23.333851 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-xpxv5_27538d1e-992b-4651-a2d2-7a7ecf442806/console-operator/0.log" Jul 17 07:12:23.333951 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:23.333884 2568 generic.go:356] "Generic (PLEG): container finished" podID="27538d1e-992b-4651-a2d2-7a7ecf442806" containerID="523b5f89853141c1ea20dd605c926c90333341e3bb767d7fa834a440ab9084a7" exitCode=255 Jul 17 07:12:23.333951 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:23.333916 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-xpxv5" event={"ID":"27538d1e-992b-4651-a2d2-7a7ecf442806","Type":"ContainerDied","Data":"523b5f89853141c1ea20dd605c926c90333341e3bb767d7fa834a440ab9084a7"} Jul 17 07:12:23.334040 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:23.333974 2568 scope.go:117] "RemoveContainer" containerID="547a3aab6f47d694148ce95f82be7838875604325392113fe0c28c84f85dea4d" Jul 17 07:12:23.334169 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:23.334151 2568 scope.go:117] "RemoveContainer" containerID="523b5f89853141c1ea20dd605c926c90333341e3bb767d7fa834a440ab9084a7" Jul 17 07:12:23.334379 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:23.334354 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-xpxv5_openshift-console-operator(27538d1e-992b-4651-a2d2-7a7ecf442806)\"" pod="openshift-console-operator/console-operator-565b7b569d-xpxv5" podUID="27538d1e-992b-4651-a2d2-7a7ecf442806" Jul 17 07:12:23.813569 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:23.813531 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9a4455d4-a377-4848-a454-54f3710b4a02-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-khqt9\" (UID: \"9a4455d4-a377-4848-a454-54f3710b4a02\") " pod="openshift-insights/insights-runtime-extractor-khqt9" Jul 17 07:12:23.813569 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:23.813570 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9a4455d4-a377-4848-a454-54f3710b4a02-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-khqt9\" (UID: \"9a4455d4-a377-4848-a454-54f3710b4a02\") " pod="openshift-insights/insights-runtime-extractor-khqt9" Jul 17 07:12:23.813778 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:23.813661 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 17 07:12:23.813778 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:23.813731 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9a4455d4-a377-4848-a454-54f3710b4a02-kube-rbac-proxy-cm podName:9a4455d4-a377-4848-a454-54f3710b4a02 nodeName:}" failed. No retries permitted until 2026-07-17 07:12:24.813715241 +0000 UTC m=+49.298404196 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9a4455d4-a377-4848-a454-54f3710b4a02-kube-rbac-proxy-cm") pod "insights-runtime-extractor-khqt9" (UID: "9a4455d4-a377-4848-a454-54f3710b4a02") : configmap "kube-rbac-proxy" not found Jul 17 07:12:23.813778 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:23.813731 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 17 07:12:23.813778 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:23.813760 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a4455d4-a377-4848-a454-54f3710b4a02-insights-runtime-extractor-tls podName:9a4455d4-a377-4848-a454-54f3710b4a02 nodeName:}" failed. No retries permitted until 2026-07-17 07:12:24.813753919 +0000 UTC m=+49.298442871 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9a4455d4-a377-4848-a454-54f3710b4a02-insights-runtime-extractor-tls") pod "insights-runtime-extractor-khqt9" (UID: "9a4455d4-a377-4848-a454-54f3710b4a02") : secret "insights-runtime-extractor-tls" not found Jul 17 07:12:24.338480 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:24.338452 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-xpxv5_27538d1e-992b-4651-a2d2-7a7ecf442806/console-operator/1.log" Jul 17 07:12:24.338843 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:24.338754 2568 scope.go:117] "RemoveContainer" containerID="523b5f89853141c1ea20dd605c926c90333341e3bb767d7fa834a440ab9084a7" Jul 17 07:12:24.338914 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:24.338894 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-xpxv5_openshift-console-operator(27538d1e-992b-4651-a2d2-7a7ecf442806)\"" pod="openshift-console-operator/console-operator-565b7b569d-xpxv5" podUID="27538d1e-992b-4651-a2d2-7a7ecf442806" Jul 17 07:12:24.519809 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:24.519775 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/254d445f-a7c6-49e6-a9a6-51b738ccdab2-metrics-certs\") pod \"router-default-6784546c7b-l47vf\" (UID: \"254d445f-a7c6-49e6-a9a6-51b738ccdab2\") " pod="openshift-ingress/router-default-6784546c7b-l47vf" Jul 17 07:12:24.520019 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:24.519876 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 17 07:12:24.520019 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:24.519955 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/254d445f-a7c6-49e6-a9a6-51b738ccdab2-metrics-certs podName:254d445f-a7c6-49e6-a9a6-51b738ccdab2 nodeName:}" failed. No retries permitted until 2026-07-17 07:12:32.519934803 +0000 UTC m=+57.004623775 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/254d445f-a7c6-49e6-a9a6-51b738ccdab2-metrics-certs") pod "router-default-6784546c7b-l47vf" (UID: "254d445f-a7c6-49e6-a9a6-51b738ccdab2") : secret "router-metrics-certs-default" not found Jul 17 07:12:24.520019 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:24.520013 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2b81b5e5-a313-4e9c-87e6-4b0242f46a78-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-z6kdb\" (UID: \"2b81b5e5-a313-4e9c-87e6-4b0242f46a78\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-z6kdb" Jul 17 07:12:24.520198 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:24.520047 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/254d445f-a7c6-49e6-a9a6-51b738ccdab2-service-ca-bundle\") pod \"router-default-6784546c7b-l47vf\" (UID: \"254d445f-a7c6-49e6-a9a6-51b738ccdab2\") " pod="openshift-ingress/router-default-6784546c7b-l47vf" Jul 17 07:12:24.520198 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:24.520154 2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 17 07:12:24.520198 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:24.520185 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/254d445f-a7c6-49e6-a9a6-51b738ccdab2-service-ca-bundle podName:254d445f-a7c6-49e6-a9a6-51b738ccdab2 nodeName:}" failed. No retries permitted until 2026-07-17 07:12:32.520172064 +0000 UTC m=+57.004861020 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/254d445f-a7c6-49e6-a9a6-51b738ccdab2-service-ca-bundle") pod "router-default-6784546c7b-l47vf" (UID: "254d445f-a7c6-49e6-a9a6-51b738ccdab2") : configmap references non-existent config key: service-ca.crt Jul 17 07:12:24.520368 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:24.520231 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b81b5e5-a313-4e9c-87e6-4b0242f46a78-samples-operator-tls podName:2b81b5e5-a313-4e9c-87e6-4b0242f46a78 nodeName:}" failed. No retries permitted until 2026-07-17 07:12:32.520212654 +0000 UTC m=+57.004901609 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2b81b5e5-a313-4e9c-87e6-4b0242f46a78-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-z6kdb" (UID: "2b81b5e5-a313-4e9c-87e6-4b0242f46a78") : secret "samples-operator-tls" not found Jul 17 07:12:24.620790 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:24.620762 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ce00a54-15e9-42b7-b3a8-9e58d1101558-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-xr6pg\" (UID: \"2ce00a54-15e9-42b7-b3a8-9e58d1101558\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xr6pg" Jul 17 07:12:24.620918 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:24.620846 2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 17 07:12:24.620918 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:24.620895 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ce00a54-15e9-42b7-b3a8-9e58d1101558-cluster-monitoring-operator-tls podName:2ce00a54-15e9-42b7-b3a8-9e58d1101558 nodeName:}" failed. No retries permitted until 2026-07-17 07:12:32.620884338 +0000 UTC m=+57.105573290 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2ce00a54-15e9-42b7-b3a8-9e58d1101558-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-xr6pg" (UID: "2ce00a54-15e9-42b7-b3a8-9e58d1101558") : secret "cluster-monitoring-operator-tls" not found Jul 17 07:12:24.795416 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:24.795393 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-t6g42_6a12d3cc-3a1a-4241-92b6-7d6198bc33c3/dns-node-resolver/0.log" Jul 17 07:12:24.822619 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:24.822594 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9a4455d4-a377-4848-a454-54f3710b4a02-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-khqt9\" (UID: \"9a4455d4-a377-4848-a454-54f3710b4a02\") " pod="openshift-insights/insights-runtime-extractor-khqt9" Jul 17 07:12:24.822733 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:24.822625 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9a4455d4-a377-4848-a454-54f3710b4a02-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-khqt9\" (UID: \"9a4455d4-a377-4848-a454-54f3710b4a02\") " pod="openshift-insights/insights-runtime-extractor-khqt9" Jul 17 07:12:24.822777 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:24.822746 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 17 07:12:24.822809 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:24.822800 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a4455d4-a377-4848-a454-54f3710b4a02-insights-runtime-extractor-tls podName:9a4455d4-a377-4848-a454-54f3710b4a02 nodeName:}" failed. No retries permitted until 2026-07-17 07:12:26.822784068 +0000 UTC m=+51.307473025 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9a4455d4-a377-4848-a454-54f3710b4a02-insights-runtime-extractor-tls") pod "insights-runtime-extractor-khqt9" (UID: "9a4455d4-a377-4848-a454-54f3710b4a02") : secret "insights-runtime-extractor-tls" not found Jul 17 07:12:24.822855 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:24.822819 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 17 07:12:24.822939 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:24.822857 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9a4455d4-a377-4848-a454-54f3710b4a02-kube-rbac-proxy-cm podName:9a4455d4-a377-4848-a454-54f3710b4a02 nodeName:}" failed. No retries permitted until 2026-07-17 07:12:26.822845735 +0000 UTC m=+51.307534702 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9a4455d4-a377-4848-a454-54f3710b4a02-kube-rbac-proxy-cm") pod "insights-runtime-extractor-khqt9" (UID: "9a4455d4-a377-4848-a454-54f3710b4a02") : configmap "kube-rbac-proxy" not found Jul 17 07:12:25.393658 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:25.393625 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-cw57w_3bc6540c-1f26-4a5e-9f66-e3480272699d/node-ca/0.log" Jul 17 07:12:26.192887 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:26.192861 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2n5zd_2df1cbe7-b30d-450f-b2e8-c3f4e10d00d5/serve-healthcheck-canary/0.log" Jul 17 07:12:26.839815 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:26.839762 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9a4455d4-a377-4848-a454-54f3710b4a02-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-khqt9\" (UID: \"9a4455d4-a377-4848-a454-54f3710b4a02\") " pod="openshift-insights/insights-runtime-extractor-khqt9" Jul 17 07:12:26.839815 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:26.839815 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9a4455d4-a377-4848-a454-54f3710b4a02-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-khqt9\" (UID: \"9a4455d4-a377-4848-a454-54f3710b4a02\") " pod="openshift-insights/insights-runtime-extractor-khqt9" Jul 17 07:12:26.840321 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:26.839907 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 17 07:12:26.840321 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:26.839962 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a4455d4-a377-4848-a454-54f3710b4a02-insights-runtime-extractor-tls podName:9a4455d4-a377-4848-a454-54f3710b4a02 nodeName:}" failed. No retries permitted until 2026-07-17 07:12:30.839949591 +0000 UTC m=+55.324638544 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9a4455d4-a377-4848-a454-54f3710b4a02-insights-runtime-extractor-tls") pod "insights-runtime-extractor-khqt9" (UID: "9a4455d4-a377-4848-a454-54f3710b4a02") : secret "insights-runtime-extractor-tls" not found Jul 17 07:12:26.840321 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:26.840263 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9a4455d4-a377-4848-a454-54f3710b4a02-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-khqt9\" (UID: \"9a4455d4-a377-4848-a454-54f3710b4a02\") " pod="openshift-insights/insights-runtime-extractor-khqt9" Jul 17 07:12:26.990474 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:26.990439 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-xpxv5" Jul 17 07:12:26.990607 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:26.990515 2568 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-xpxv5" Jul 17 07:12:26.990851 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:26.990837 2568 scope.go:117] "RemoveContainer" containerID="523b5f89853141c1ea20dd605c926c90333341e3bb767d7fa834a440ab9084a7" Jul 17 07:12:26.991004 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:26.990985 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-xpxv5_openshift-console-operator(27538d1e-992b-4651-a2d2-7a7ecf442806)\"" pod="openshift-console-operator/console-operator-565b7b569d-xpxv5" podUID="27538d1e-992b-4651-a2d2-7a7ecf442806" Jul 17 07:12:27.142337 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:27.142311 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b1d8b7b7-173b-4e59-acce-7fbe2b1f5465-metrics-tls\") pod \"dns-default-vc255\" (UID: \"b1d8b7b7-173b-4e59-acce-7fbe2b1f5465\") " pod="openshift-dns/dns-default-vc255" Jul 17 07:12:27.142486 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:27.142424 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0161fccb-b49c-4592-8a96-b79d237fe749-registry-tls\") pod \"image-registry-6675666ffd-gnf79\" (UID: \"0161fccb-b49c-4592-8a96-b79d237fe749\") " pod="openshift-image-registry/image-registry-6675666ffd-gnf79" Jul 17 07:12:27.142486 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:27.142437 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 17 07:12:27.142553 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:27.142528 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 17 07:12:27.142553 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:27.142543 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6675666ffd-gnf79: secret "image-registry-tls" not found Jul 17 07:12:27.142618 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:27.142563 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1d8b7b7-173b-4e59-acce-7fbe2b1f5465-metrics-tls podName:b1d8b7b7-173b-4e59-acce-7fbe2b1f5465 nodeName:}" failed. No retries permitted until 2026-07-17 07:12:43.142541307 +0000 UTC m=+67.627230272 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b1d8b7b7-173b-4e59-acce-7fbe2b1f5465-metrics-tls") pod "dns-default-vc255" (UID: "b1d8b7b7-173b-4e59-acce-7fbe2b1f5465") : secret "dns-default-metrics-tls" not found Jul 17 07:12:27.142618 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:27.142599 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0161fccb-b49c-4592-8a96-b79d237fe749-registry-tls podName:0161fccb-b49c-4592-8a96-b79d237fe749 nodeName:}" failed. No retries permitted until 2026-07-17 07:12:43.142584809 +0000 UTC m=+67.627273784 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0161fccb-b49c-4592-8a96-b79d237fe749-registry-tls") pod "image-registry-6675666ffd-gnf79" (UID: "0161fccb-b49c-4592-8a96-b79d237fe749") : secret "image-registry-tls" not found Jul 17 07:12:30.874328 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:30.874279 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9a4455d4-a377-4848-a454-54f3710b4a02-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-khqt9\" (UID: \"9a4455d4-a377-4848-a454-54f3710b4a02\") " pod="openshift-insights/insights-runtime-extractor-khqt9" Jul 17 07:12:30.874774 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:30.874402 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 17 07:12:30.874774 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:30.874453 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a4455d4-a377-4848-a454-54f3710b4a02-insights-runtime-extractor-tls podName:9a4455d4-a377-4848-a454-54f3710b4a02 nodeName:}" failed. No retries permitted until 2026-07-17 07:12:38.87444037 +0000 UTC m=+63.359129322 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9a4455d4-a377-4848-a454-54f3710b4a02-insights-runtime-extractor-tls") pod "insights-runtime-extractor-khqt9" (UID: "9a4455d4-a377-4848-a454-54f3710b4a02") : secret "insights-runtime-extractor-tls" not found Jul 17 07:12:32.589275 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:32.589235 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/254d445f-a7c6-49e6-a9a6-51b738ccdab2-metrics-certs\") pod \"router-default-6784546c7b-l47vf\" (UID: \"254d445f-a7c6-49e6-a9a6-51b738ccdab2\") " pod="openshift-ingress/router-default-6784546c7b-l47vf" Jul 17 07:12:32.589779 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:32.589339 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2b81b5e5-a313-4e9c-87e6-4b0242f46a78-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-z6kdb\" (UID: \"2b81b5e5-a313-4e9c-87e6-4b0242f46a78\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-z6kdb" Jul 17 07:12:32.589779 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:32.589362 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/254d445f-a7c6-49e6-a9a6-51b738ccdab2-service-ca-bundle\") pod \"router-default-6784546c7b-l47vf\" (UID: \"254d445f-a7c6-49e6-a9a6-51b738ccdab2\") " pod="openshift-ingress/router-default-6784546c7b-l47vf" Jul 17 07:12:32.589779 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:32.589390 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 17 07:12:32.589779 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:32.589457 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/254d445f-a7c6-49e6-a9a6-51b738ccdab2-metrics-certs podName:254d445f-a7c6-49e6-a9a6-51b738ccdab2 nodeName:}" failed. No retries permitted until 2026-07-17 07:12:48.589440733 +0000 UTC m=+73.074129685 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/254d445f-a7c6-49e6-a9a6-51b738ccdab2-metrics-certs") pod "router-default-6784546c7b-l47vf" (UID: "254d445f-a7c6-49e6-a9a6-51b738ccdab2") : secret "router-metrics-certs-default" not found Jul 17 07:12:32.589779 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:32.589470 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/254d445f-a7c6-49e6-a9a6-51b738ccdab2-service-ca-bundle podName:254d445f-a7c6-49e6-a9a6-51b738ccdab2 nodeName:}" failed. No retries permitted until 2026-07-17 07:12:48.589464259 +0000 UTC m=+73.074153210 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/254d445f-a7c6-49e6-a9a6-51b738ccdab2-service-ca-bundle") pod "router-default-6784546c7b-l47vf" (UID: "254d445f-a7c6-49e6-a9a6-51b738ccdab2") : configmap references non-existent config key: service-ca.crt Jul 17 07:12:32.591673 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:32.591643 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2b81b5e5-a313-4e9c-87e6-4b0242f46a78-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-z6kdb\" (UID: \"2b81b5e5-a313-4e9c-87e6-4b0242f46a78\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-z6kdb" Jul 17 07:12:32.608667 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:32.608645 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-z6kdb" Jul 17 07:12:32.689890 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:32.689855 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ce00a54-15e9-42b7-b3a8-9e58d1101558-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-xr6pg\" (UID: \"2ce00a54-15e9-42b7-b3a8-9e58d1101558\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xr6pg" Jul 17 07:12:32.690029 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:32.690010 2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 17 07:12:32.690111 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:12:32.690100 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ce00a54-15e9-42b7-b3a8-9e58d1101558-cluster-monitoring-operator-tls podName:2ce00a54-15e9-42b7-b3a8-9e58d1101558 nodeName:}" failed. No retries permitted until 2026-07-17 07:12:48.690080217 +0000 UTC m=+73.174769184 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2ce00a54-15e9-42b7-b3a8-9e58d1101558-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-xr6pg" (UID: "2ce00a54-15e9-42b7-b3a8-9e58d1101558") : secret "cluster-monitoring-operator-tls" not found Jul 17 07:12:32.740787 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:32.740757 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-z6kdb"] Jul 17 07:12:33.369925 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:33.369888 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-z6kdb" event={"ID":"2b81b5e5-a313-4e9c-87e6-4b0242f46a78","Type":"ContainerStarted","Data":"ed1447f6b2bbf30caafe3a4d5c19e6cbbbc88f53149e1251692ede0e0de2f49b"} Jul 17 07:12:35.377172 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:35.377135 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-z6kdb" event={"ID":"2b81b5e5-a313-4e9c-87e6-4b0242f46a78","Type":"ContainerStarted","Data":"2bca8f2daa399341ccb16e0791c7a2b75e5e97a3d09a3d27784a146e12429c08"} Jul 17 07:12:35.377672 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:35.377187 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-z6kdb" event={"ID":"2b81b5e5-a313-4e9c-87e6-4b0242f46a78","Type":"ContainerStarted","Data":"35876d00fc9096fe9322363a8ba1af5da00c6e72dc1a4dc3882ba711f1062560"} Jul 17 07:12:35.393964 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:35.393913 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-z6kdb" podStartSLOduration=17.825849832 podStartE2EDuration="19.393898859s" totalImagesPullingTime="1.568049027s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 07:12:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:12:35.392925489 +0000 UTC m=+59.877614455" watchObservedRunningTime="2026-07-17 07:12:35.393898859 +0000 UTC m=+59.878587811" Jul 17 07:12:38.274036 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:38.274007 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cpwz4" Jul 17 07:12:38.941325 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:38.941258 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9a4455d4-a377-4848-a454-54f3710b4a02-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-khqt9\" (UID: \"9a4455d4-a377-4848-a454-54f3710b4a02\") " pod="openshift-insights/insights-runtime-extractor-khqt9" Jul 17 07:12:38.943579 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:38.943544 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9a4455d4-a377-4848-a454-54f3710b4a02-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-khqt9\" (UID: \"9a4455d4-a377-4848-a454-54f3710b4a02\") " pod="openshift-insights/insights-runtime-extractor-khqt9" Jul 17 07:12:39.235157 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:39.235075 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-gcxnf\"" Jul 17 07:12:39.243007 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:39.242991 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-khqt9" Jul 17 07:12:39.362857 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:39.362832 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-khqt9"] Jul 17 07:12:39.365312 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:12:39.365269 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a4455d4_a377_4848_a454_54f3710b4a02.slice/crio-6ff494bc9c9c37e03a767d4dcd0370c7f8053b39526db77a466ab07216868cf4 WatchSource:0}: Error finding container 6ff494bc9c9c37e03a767d4dcd0370c7f8053b39526db77a466ab07216868cf4: Status 404 returned error can't find the container with id 6ff494bc9c9c37e03a767d4dcd0370c7f8053b39526db77a466ab07216868cf4 Jul 17 07:12:39.393948 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:39.393918 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-khqt9" event={"ID":"9a4455d4-a377-4848-a454-54f3710b4a02","Type":"ContainerStarted","Data":"6ff494bc9c9c37e03a767d4dcd0370c7f8053b39526db77a466ab07216868cf4"} Jul 17 07:12:40.758630 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:40.758588 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/deb65c16-2d4c-40f0-aa9c-4399304b7222-metrics-certs\") pod \"network-metrics-daemon-rvx7b\" (UID: \"deb65c16-2d4c-40f0-aa9c-4399304b7222\") " pod="openshift-multus/network-metrics-daemon-rvx7b" Jul 17 07:12:40.759088 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:40.758644 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2639225a-fbee-4ebd-b36f-7ad47f365e79-original-pull-secret\") pod \"global-pull-secret-syncer-4x4z6\" (UID: \"2639225a-fbee-4ebd-b36f-7ad47f365e79\") " pod="kube-system/global-pull-secret-syncer-4x4z6" Jul 17 07:12:40.764567 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:40.764539 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 17 07:12:40.764705 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:40.764642 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 17 07:12:40.772356 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:40.772333 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2639225a-fbee-4ebd-b36f-7ad47f365e79-original-pull-secret\") pod \"global-pull-secret-syncer-4x4z6\" (UID: \"2639225a-fbee-4ebd-b36f-7ad47f365e79\") " pod="kube-system/global-pull-secret-syncer-4x4z6" Jul 17 07:12:40.772474 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:40.772411 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/deb65c16-2d4c-40f0-aa9c-4399304b7222-metrics-certs\") pod \"network-metrics-daemon-rvx7b\" (UID: \"deb65c16-2d4c-40f0-aa9c-4399304b7222\") " pod="openshift-multus/network-metrics-daemon-rvx7b" Jul 17 07:12:40.809499 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:40.809472 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4x4z6" Jul 17 07:12:40.860092 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:40.860058 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wmqr\" (UniqueName: \"kubernetes.io/projected/ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376-kube-api-access-9wmqr\") pod \"network-check-target-dtjs7\" (UID: \"ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376\") " pod="openshift-network-diagnostics/network-check-target-dtjs7" Jul 17 07:12:40.862949 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:40.862862 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 17 07:12:40.874597 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:40.874575 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 17 07:12:40.885027 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:40.885002 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wmqr\" (UniqueName: \"kubernetes.io/projected/ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376-kube-api-access-9wmqr\") pod \"network-check-target-dtjs7\" (UID: \"ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376\") " pod="openshift-network-diagnostics/network-check-target-dtjs7" Jul 17 07:12:40.906962 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:40.906941 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-s4jgz\"" Jul 17 07:12:40.915245 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:40.914055 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rvx7b" Jul 17 07:12:40.950039 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:40.950008 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4x4z6"] Jul 17 07:12:40.953130 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:12:40.953093 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2639225a_fbee_4ebd_b36f_7ad47f365e79.slice/crio-0c05c5c4f35cc0bd662857720c65a5320c04587ce78e58b03f9c69390418f941 WatchSource:0}: Error finding container 0c05c5c4f35cc0bd662857720c65a5320c04587ce78e58b03f9c69390418f941: Status 404 returned error can't find the container with id 0c05c5c4f35cc0bd662857720c65a5320c04587ce78e58b03f9c69390418f941 Jul 17 07:12:40.994195 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:40.994170 2568 scope.go:117] "RemoveContainer" containerID="523b5f89853141c1ea20dd605c926c90333341e3bb767d7fa834a440ab9084a7" Jul 17 07:12:41.057230 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:41.057201 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rvx7b"] Jul 17 07:12:41.060643 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:12:41.060617 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddeb65c16_2d4c_40f0_aa9c_4399304b7222.slice/crio-2a245f3842a247abdc3d6b7bb355f214d15d115d176e00156d5c4cf0386071d8 WatchSource:0}: Error finding container 2a245f3842a247abdc3d6b7bb355f214d15d115d176e00156d5c4cf0386071d8: Status 404 returned error can't find the container with id 2a245f3842a247abdc3d6b7bb355f214d15d115d176e00156d5c4cf0386071d8 Jul 17 07:12:41.106960 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:41.106925 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-c28tv\"" Jul 17 07:12:41.115428 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:41.115406 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dtjs7" Jul 17 07:12:41.256105 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:41.256059 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-dtjs7"] Jul 17 07:12:41.259038 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:12:41.259001 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca9d4cc1_0fb9_43c9_91b2_a0a84e2b8376.slice/crio-1b113baf6260216f0605ce7f9b825ad51960e5cb94cd641f38989dd392403740 WatchSource:0}: Error finding container 1b113baf6260216f0605ce7f9b825ad51960e5cb94cd641f38989dd392403740: Status 404 returned error can't find the container with id 1b113baf6260216f0605ce7f9b825ad51960e5cb94cd641f38989dd392403740 Jul 17 07:12:41.403962 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:41.403927 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rvx7b" event={"ID":"deb65c16-2d4c-40f0-aa9c-4399304b7222","Type":"ContainerStarted","Data":"2a245f3842a247abdc3d6b7bb355f214d15d115d176e00156d5c4cf0386071d8"} Jul 17 07:12:41.406120 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:41.406096 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-xpxv5_27538d1e-992b-4651-a2d2-7a7ecf442806/console-operator/1.log" Jul 17 07:12:41.406268 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:41.406197 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-xpxv5" event={"ID":"27538d1e-992b-4651-a2d2-7a7ecf442806","Type":"ContainerStarted","Data":"6fc2f8d5dd214c4010e80b60401f08fb25539bedeb092d4b123c8880d97aa3ee"} Jul 17 07:12:41.406704 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:41.406487 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-xpxv5" Jul 17 07:12:41.407755 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:41.407714 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-dtjs7" event={"ID":"ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376","Type":"ContainerStarted","Data":"1b113baf6260216f0605ce7f9b825ad51960e5cb94cd641f38989dd392403740"} Jul 17 07:12:41.409103 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:41.409081 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4x4z6" event={"ID":"2639225a-fbee-4ebd-b36f-7ad47f365e79","Type":"ContainerStarted","Data":"0c05c5c4f35cc0bd662857720c65a5320c04587ce78e58b03f9c69390418f941"} Jul 17 07:12:41.425437 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:41.425039 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-xpxv5" podStartSLOduration=21.12064162 podStartE2EDuration="25.425012363s" totalImagesPullingTime="4.304370743s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 07:12:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:12:41.424515624 +0000 UTC m=+65.909204598" watchObservedRunningTime="2026-07-17 07:12:41.425012363 +0000 UTC m=+65.909701338" Jul 17 07:12:41.474844 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:41.474811 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-xpxv5" Jul 17 07:12:43.181945 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:43.181734 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b1d8b7b7-173b-4e59-acce-7fbe2b1f5465-metrics-tls\") pod \"dns-default-vc255\" (UID: \"b1d8b7b7-173b-4e59-acce-7fbe2b1f5465\") " pod="openshift-dns/dns-default-vc255" Jul 17 07:12:43.181945 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:43.181848 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0161fccb-b49c-4592-8a96-b79d237fe749-registry-tls\") pod \"image-registry-6675666ffd-gnf79\" (UID: \"0161fccb-b49c-4592-8a96-b79d237fe749\") " pod="openshift-image-registry/image-registry-6675666ffd-gnf79" Jul 17 07:12:43.184720 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:43.184695 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b1d8b7b7-173b-4e59-acce-7fbe2b1f5465-metrics-tls\") pod \"dns-default-vc255\" (UID: \"b1d8b7b7-173b-4e59-acce-7fbe2b1f5465\") " pod="openshift-dns/dns-default-vc255" Jul 17 07:12:43.185687 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:43.185660 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0161fccb-b49c-4592-8a96-b79d237fe749-registry-tls\") pod \"image-registry-6675666ffd-gnf79\" (UID: \"0161fccb-b49c-4592-8a96-b79d237fe749\") " pod="openshift-image-registry/image-registry-6675666ffd-gnf79" Jul 17 07:12:43.408799 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:43.408764 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-lwnzz\"" Jul 17 07:12:43.416656 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:43.416614 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6675666ffd-gnf79" Jul 17 07:12:43.419379 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:43.419168 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-kjdqb\"" Jul 17 07:12:43.427318 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:43.427257 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vc255" Jul 17 07:12:48.108001 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:48.107720 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6675666ffd-gnf79"] Jul 17 07:12:48.112182 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:12:48.112159 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0161fccb_b49c_4592_8a96_b79d237fe749.slice/crio-d39e23320c8138a671bbb03a947796fa9489690532c10f2de4b89861a090b66e WatchSource:0}: Error finding container d39e23320c8138a671bbb03a947796fa9489690532c10f2de4b89861a090b66e: Status 404 returned error can't find the container with id d39e23320c8138a671bbb03a947796fa9489690532c10f2de4b89861a090b66e Jul 17 07:12:48.310378 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:48.310345 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vc255"] Jul 17 07:12:48.313168 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:12:48.313138 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1d8b7b7_173b_4e59_acce_7fbe2b1f5465.slice/crio-b201c5f12703bde30b4d5771475213c3aa78fad53aaf7a14822f4d7917ed4128 WatchSource:0}: Error finding container b201c5f12703bde30b4d5771475213c3aa78fad53aaf7a14822f4d7917ed4128: Status 404 returned error can't find the container with id b201c5f12703bde30b4d5771475213c3aa78fad53aaf7a14822f4d7917ed4128 Jul 17 07:12:48.441369 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:48.441334 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-dtjs7" event={"ID":"ca9d4cc1-0fb9-43c9-91b2-a0a84e2b8376","Type":"ContainerStarted","Data":"6f7c79803e1d66bc4262ac2d0158ba2e18da08c7c9b3c55171329ba09365a02a"} Jul 17 07:12:48.441551 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:48.441425 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-dtjs7" Jul 17 07:12:48.442963 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:48.442937 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4x4z6" event={"ID":"2639225a-fbee-4ebd-b36f-7ad47f365e79","Type":"ContainerStarted","Data":"29fbd6d62a9d17d712eede62348e1109fa1500d622f6ecf85efdf1a334040ed6"} Jul 17 07:12:48.444530 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:48.444487 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6675666ffd-gnf79" event={"ID":"0161fccb-b49c-4592-8a96-b79d237fe749","Type":"ContainerStarted","Data":"b33b9d6f46c4e625e542828f0056b377e022f551e98db0dfbeaa4af27f9429a3"} Jul 17 07:12:48.444669 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:48.444547 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6675666ffd-gnf79" event={"ID":"0161fccb-b49c-4592-8a96-b79d237fe749","Type":"ContainerStarted","Data":"d39e23320c8138a671bbb03a947796fa9489690532c10f2de4b89861a090b66e"} Jul 17 07:12:48.444669 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:48.444581 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6675666ffd-gnf79" Jul 17 07:12:48.446004 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:48.445980 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-khqt9" event={"ID":"9a4455d4-a377-4848-a454-54f3710b4a02","Type":"ContainerStarted","Data":"7a13095584d477d9ef5fb66024004a3cbec5429b2b72c631c2bee2e900124960"} Jul 17 07:12:48.447702 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:48.447681 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rvx7b" event={"ID":"deb65c16-2d4c-40f0-aa9c-4399304b7222","Type":"ContainerStarted","Data":"0983afe5daa38f813fcfe2d3cee11cec9f350033370049144d2c3b591ea1d642"} Jul 17 07:12:48.447827 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:48.447724 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rvx7b" event={"ID":"deb65c16-2d4c-40f0-aa9c-4399304b7222","Type":"ContainerStarted","Data":"5d83a95e7009e0b267d426c27cbe9ed5385b1c5e63a6d74df6c27d92433e726a"} Jul 17 07:12:48.448752 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:48.448724 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vc255" event={"ID":"b1d8b7b7-173b-4e59-acce-7fbe2b1f5465","Type":"ContainerStarted","Data":"b201c5f12703bde30b4d5771475213c3aa78fad53aaf7a14822f4d7917ed4128"} Jul 17 07:12:48.457226 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:48.457152 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-dtjs7" podStartSLOduration=65.771424571 podStartE2EDuration="1m12.45714032s" totalImagesPullingTime="6.685715749s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 07:11:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:12:48.45616692 +0000 UTC m=+72.940855895" watchObservedRunningTime="2026-07-17 07:12:48.45714032 +0000 UTC m=+72.941829293" Jul 17 07:12:48.487139 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:48.487096 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-4x4z6" podStartSLOduration=65.495323951 podStartE2EDuration="1m12.487079583s" totalImagesPullingTime="6.991755632s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 07:11:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:12:48.470818134 +0000 UTC m=+72.955507108" watchObservedRunningTime="2026-07-17 07:12:48.487079583 +0000 UTC m=+72.971768551" Jul 17 07:12:48.487826 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:48.487772 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-rvx7b" podStartSLOduration=65.603641781 podStartE2EDuration="1m12.487762158s" totalImagesPullingTime="6.884120377s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 07:11:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:12:48.48656522 +0000 UTC m=+72.971254185" watchObservedRunningTime="2026-07-17 07:12:48.487762158 +0000 UTC m=+72.972451139" Jul 17 07:12:48.506932 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:48.506883 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6675666ffd-gnf79" podStartSLOduration=72.50686478 podStartE2EDuration="1m12.50686478s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 07:11:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:12:48.505558923 +0000 UTC m=+72.990247896" watchObservedRunningTime="2026-07-17 07:12:48.50686478 +0000 UTC m=+72.991553755" Jul 17 07:12:48.632531 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:48.632500 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/254d445f-a7c6-49e6-a9a6-51b738ccdab2-metrics-certs\") pod \"router-default-6784546c7b-l47vf\" (UID: \"254d445f-a7c6-49e6-a9a6-51b738ccdab2\") " pod="openshift-ingress/router-default-6784546c7b-l47vf" Jul 17 07:12:48.632712 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:48.632561 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/254d445f-a7c6-49e6-a9a6-51b738ccdab2-service-ca-bundle\") pod \"router-default-6784546c7b-l47vf\" (UID: \"254d445f-a7c6-49e6-a9a6-51b738ccdab2\") " pod="openshift-ingress/router-default-6784546c7b-l47vf" Jul 17 07:12:48.633253 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:48.633224 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/254d445f-a7c6-49e6-a9a6-51b738ccdab2-service-ca-bundle\") pod \"router-default-6784546c7b-l47vf\" (UID: \"254d445f-a7c6-49e6-a9a6-51b738ccdab2\") " pod="openshift-ingress/router-default-6784546c7b-l47vf" Jul 17 07:12:48.635143 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:48.635107 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/254d445f-a7c6-49e6-a9a6-51b738ccdab2-metrics-certs\") pod \"router-default-6784546c7b-l47vf\" (UID: \"254d445f-a7c6-49e6-a9a6-51b738ccdab2\") " pod="openshift-ingress/router-default-6784546c7b-l47vf" Jul 17 07:12:48.733508 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:48.733309 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ce00a54-15e9-42b7-b3a8-9e58d1101558-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-xr6pg\" (UID: \"2ce00a54-15e9-42b7-b3a8-9e58d1101558\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xr6pg" Jul 17 07:12:48.737054 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:48.736981 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ce00a54-15e9-42b7-b3a8-9e58d1101558-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-xr6pg\" (UID: \"2ce00a54-15e9-42b7-b3a8-9e58d1101558\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xr6pg" Jul 17 07:12:48.833109 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:48.833074 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-djw4q\"" Jul 17 07:12:48.841371 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:48.841337 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6784546c7b-l47vf" Jul 17 07:12:48.915322 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:48.915273 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-78zzm\"" Jul 17 07:12:48.924708 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:48.924553 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xr6pg" Jul 17 07:12:49.038266 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:49.038235 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6784546c7b-l47vf"] Jul 17 07:12:49.045488 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:12:49.045448 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod254d445f_a7c6_49e6_a9a6_51b738ccdab2.slice/crio-5978107b6274120f057852f41e346c84fdf857b58da8a8a9c9398ecc49ea202e WatchSource:0}: Error finding container 5978107b6274120f057852f41e346c84fdf857b58da8a8a9c9398ecc49ea202e: Status 404 returned error can't find the container with id 5978107b6274120f057852f41e346c84fdf857b58da8a8a9c9398ecc49ea202e Jul 17 07:12:49.100886 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:49.099620 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xr6pg"] Jul 17 07:12:49.106494 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:12:49.106460 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ce00a54_15e9_42b7_b3a8_9e58d1101558.slice/crio-73f885d5ed1344ec4186f91cbf29dd6ab81754a61e9c534245f9411434c92c4e WatchSource:0}: Error finding container 73f885d5ed1344ec4186f91cbf29dd6ab81754a61e9c534245f9411434c92c4e: Status 404 returned error can't find the container with id 73f885d5ed1344ec4186f91cbf29dd6ab81754a61e9c534245f9411434c92c4e Jul 17 07:12:49.455455 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:49.455417 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xr6pg" event={"ID":"2ce00a54-15e9-42b7-b3a8-9e58d1101558","Type":"ContainerStarted","Data":"73f885d5ed1344ec4186f91cbf29dd6ab81754a61e9c534245f9411434c92c4e"} Jul 17 07:12:49.457580 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:49.457545 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6784546c7b-l47vf" event={"ID":"254d445f-a7c6-49e6-a9a6-51b738ccdab2","Type":"ContainerStarted","Data":"056fa8632086e30cf98416e53d9b452ab827f2888d2ef98a419930c7881c4287"} Jul 17 07:12:49.457715 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:49.457600 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6784546c7b-l47vf" event={"ID":"254d445f-a7c6-49e6-a9a6-51b738ccdab2","Type":"ContainerStarted","Data":"5978107b6274120f057852f41e346c84fdf857b58da8a8a9c9398ecc49ea202e"} Jul 17 07:12:49.479301 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:49.479111 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-6784546c7b-l47vf" podStartSLOduration=33.47909275 podStartE2EDuration="33.47909275s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 07:12:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:12:49.477914225 +0000 UTC m=+73.962603199" watchObservedRunningTime="2026-07-17 07:12:49.47909275 +0000 UTC m=+73.963781726" Jul 17 07:12:49.842117 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:49.842022 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-6784546c7b-l47vf" Jul 17 07:12:49.845343 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:49.845314 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-6784546c7b-l47vf" Jul 17 07:12:50.461668 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:50.461636 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-6784546c7b-l47vf" Jul 17 07:12:50.463104 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:50.463080 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-6784546c7b-l47vf" Jul 17 07:12:51.715349 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:51.715315 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65fcb7c85f-d4wct"] Jul 17 07:12:51.722333 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:51.722311 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65fcb7c85f-d4wct" Jul 17 07:12:51.726393 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:51.726369 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 17 07:12:51.726792 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:51.726770 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 17 07:12:51.726900 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:51.726852 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 17 07:12:51.727204 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:51.727184 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 17 07:12:51.727307 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:51.727206 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 17 07:12:51.727307 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:51.727207 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 17 07:12:51.729249 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:51.729194 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 17 07:12:51.745869 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:51.745845 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65fcb7c85f-d4wct"] Jul 17 07:12:51.862690 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:51.862654 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/9def99f9-2e41-4725-9f28-ba63f7990920-additional-service-ca\") pod \"cluster-proxy-proxy-agent-65fcb7c85f-d4wct\" (UID: \"9def99f9-2e41-4725-9f28-ba63f7990920\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65fcb7c85f-d4wct" Jul 17 07:12:51.862890 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:51.862694 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vf7f7\" (UniqueName: \"kubernetes.io/projected/9def99f9-2e41-4725-9f28-ba63f7990920-kube-api-access-vf7f7\") pod \"cluster-proxy-proxy-agent-65fcb7c85f-d4wct\" (UID: \"9def99f9-2e41-4725-9f28-ba63f7990920\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65fcb7c85f-d4wct" Jul 17 07:12:51.862890 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:51.862725 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/9def99f9-2e41-4725-9f28-ba63f7990920-hub\") pod \"cluster-proxy-proxy-agent-65fcb7c85f-d4wct\" (UID: \"9def99f9-2e41-4725-9f28-ba63f7990920\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65fcb7c85f-d4wct" Jul 17 07:12:51.862890 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:51.862758 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/9def99f9-2e41-4725-9f28-ba63f7990920-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-65fcb7c85f-d4wct\" (UID: \"9def99f9-2e41-4725-9f28-ba63f7990920\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65fcb7c85f-d4wct" Jul 17 07:12:51.862890 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:51.862870 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/9def99f9-2e41-4725-9f28-ba63f7990920-ca\") pod \"cluster-proxy-proxy-agent-65fcb7c85f-d4wct\" (UID: \"9def99f9-2e41-4725-9f28-ba63f7990920\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65fcb7c85f-d4wct" Jul 17 07:12:51.863102 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:51.862909 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9def99f9-2e41-4725-9f28-ba63f7990920-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-65fcb7c85f-d4wct\" (UID: \"9def99f9-2e41-4725-9f28-ba63f7990920\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65fcb7c85f-d4wct" Jul 17 07:12:51.964126 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:51.964091 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/9def99f9-2e41-4725-9f28-ba63f7990920-additional-service-ca\") pod \"cluster-proxy-proxy-agent-65fcb7c85f-d4wct\" (UID: \"9def99f9-2e41-4725-9f28-ba63f7990920\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65fcb7c85f-d4wct" Jul 17 07:12:51.964126 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:51.964136 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vf7f7\" (UniqueName: \"kubernetes.io/projected/9def99f9-2e41-4725-9f28-ba63f7990920-kube-api-access-vf7f7\") pod \"cluster-proxy-proxy-agent-65fcb7c85f-d4wct\" (UID: \"9def99f9-2e41-4725-9f28-ba63f7990920\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65fcb7c85f-d4wct" Jul 17 07:12:51.964385 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:51.964173 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/9def99f9-2e41-4725-9f28-ba63f7990920-hub\") pod \"cluster-proxy-proxy-agent-65fcb7c85f-d4wct\" (UID: \"9def99f9-2e41-4725-9f28-ba63f7990920\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65fcb7c85f-d4wct" Jul 17 07:12:51.964385 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:51.964203 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/9def99f9-2e41-4725-9f28-ba63f7990920-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-65fcb7c85f-d4wct\" (UID: \"9def99f9-2e41-4725-9f28-ba63f7990920\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65fcb7c85f-d4wct" Jul 17 07:12:51.964385 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:51.964256 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/9def99f9-2e41-4725-9f28-ba63f7990920-ca\") pod \"cluster-proxy-proxy-agent-65fcb7c85f-d4wct\" (UID: \"9def99f9-2e41-4725-9f28-ba63f7990920\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65fcb7c85f-d4wct" Jul 17 07:12:51.964385 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:51.964279 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9def99f9-2e41-4725-9f28-ba63f7990920-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-65fcb7c85f-d4wct\" (UID: \"9def99f9-2e41-4725-9f28-ba63f7990920\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65fcb7c85f-d4wct" Jul 17 07:12:51.965072 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:51.965038 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/9def99f9-2e41-4725-9f28-ba63f7990920-additional-service-ca\") pod \"cluster-proxy-proxy-agent-65fcb7c85f-d4wct\" (UID: \"9def99f9-2e41-4725-9f28-ba63f7990920\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65fcb7c85f-d4wct" Jul 17 07:12:51.967835 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:51.967228 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/9def99f9-2e41-4725-9f28-ba63f7990920-hub\") pod \"cluster-proxy-proxy-agent-65fcb7c85f-d4wct\" (UID: \"9def99f9-2e41-4725-9f28-ba63f7990920\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65fcb7c85f-d4wct" Jul 17 07:12:51.967835 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:51.967253 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9def99f9-2e41-4725-9f28-ba63f7990920-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-65fcb7c85f-d4wct\" (UID: \"9def99f9-2e41-4725-9f28-ba63f7990920\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65fcb7c85f-d4wct" Jul 17 07:12:51.967835 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:51.967411 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/9def99f9-2e41-4725-9f28-ba63f7990920-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-65fcb7c85f-d4wct\" (UID: \"9def99f9-2e41-4725-9f28-ba63f7990920\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65fcb7c85f-d4wct" Jul 17 07:12:51.967835 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:51.967643 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/9def99f9-2e41-4725-9f28-ba63f7990920-ca\") pod \"cluster-proxy-proxy-agent-65fcb7c85f-d4wct\" (UID: \"9def99f9-2e41-4725-9f28-ba63f7990920\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65fcb7c85f-d4wct" Jul 17 07:12:51.971891 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:51.971871 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vf7f7\" (UniqueName: \"kubernetes.io/projected/9def99f9-2e41-4725-9f28-ba63f7990920-kube-api-access-vf7f7\") pod \"cluster-proxy-proxy-agent-65fcb7c85f-d4wct\" (UID: \"9def99f9-2e41-4725-9f28-ba63f7990920\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65fcb7c85f-d4wct" Jul 17 07:12:52.043237 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:52.043200 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65fcb7c85f-d4wct" Jul 17 07:12:53.099774 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:53.099695 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65fcb7c85f-d4wct"] Jul 17 07:12:53.101926 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:12:53.101898 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9def99f9_2e41_4725_9f28_ba63f7990920.slice/crio-faafce666e0d992e534cea62d068e5bb2ac24339a0dadd626b8b78517ccf2467 WatchSource:0}: Error finding container faafce666e0d992e534cea62d068e5bb2ac24339a0dadd626b8b78517ccf2467: Status 404 returned error can't find the container with id faafce666e0d992e534cea62d068e5bb2ac24339a0dadd626b8b78517ccf2467 Jul 17 07:12:53.474666 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:53.474634 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xr6pg" event={"ID":"2ce00a54-15e9-42b7-b3a8-9e58d1101558","Type":"ContainerStarted","Data":"ad0352d2020f7e512242fddac57d1a6a4befde6033afc0d60122d97b5b9271df"} Jul 17 07:12:53.476582 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:53.476558 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-khqt9" event={"ID":"9a4455d4-a377-4848-a454-54f3710b4a02","Type":"ContainerStarted","Data":"ee253d3767e1f43a20693989b93427e79a58df9b4a8398d3ba27e18f8f3a6649"} Jul 17 07:12:53.478534 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:53.478504 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vc255" event={"ID":"b1d8b7b7-173b-4e59-acce-7fbe2b1f5465","Type":"ContainerStarted","Data":"daf920b9c567f2fd9e93349fdcdb1179624e21dbfd7a901e352eccfabf9f3e40"} Jul 17 07:12:53.478654 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:53.478560 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vc255" event={"ID":"b1d8b7b7-173b-4e59-acce-7fbe2b1f5465","Type":"ContainerStarted","Data":"825cfa6a4f38f9c7f8adb29cfd398b2c0ef00a67610406589b591dcf24dbbc7f"} Jul 17 07:12:53.478654 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:53.478592 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-vc255" Jul 17 07:12:53.479638 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:53.479616 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65fcb7c85f-d4wct" event={"ID":"9def99f9-2e41-4725-9f28-ba63f7990920","Type":"ContainerStarted","Data":"faafce666e0d992e534cea62d068e5bb2ac24339a0dadd626b8b78517ccf2467"} Jul 17 07:12:53.497258 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:53.497218 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xr6pg" podStartSLOduration=33.640389207 podStartE2EDuration="37.49720487s" totalImagesPullingTime="3.856815663s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 07:12:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:12:53.495575821 +0000 UTC m=+77.980264798" watchObservedRunningTime="2026-07-17 07:12:53.49720487 +0000 UTC m=+77.981893844" Jul 17 07:12:53.519335 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:53.519269 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-vc255" podStartSLOduration=37.907348473 podStartE2EDuration="42.519253039s" totalImagesPullingTime="4.611904566s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 07:12:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:12:53.518222838 +0000 UTC m=+78.002911812" watchObservedRunningTime="2026-07-17 07:12:53.519253039 +0000 UTC m=+78.003942048" Jul 17 07:12:59.508177 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:59.508133 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-khqt9" event={"ID":"9a4455d4-a377-4848-a454-54f3710b4a02","Type":"ContainerStarted","Data":"53e88994339b788031daad3df84ccc23a42c88d61133cdc788f284850558332a"} Jul 17 07:12:59.510107 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:59.510081 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65fcb7c85f-d4wct" event={"ID":"9def99f9-2e41-4725-9f28-ba63f7990920","Type":"ContainerStarted","Data":"a861448cbf507a0512a1a714838575621ea2d52c5b1da87c554b374e14150b6e"} Jul 17 07:12:59.510226 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:59.510118 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65fcb7c85f-d4wct" event={"ID":"9def99f9-2e41-4725-9f28-ba63f7990920","Type":"ContainerStarted","Data":"1ddde2cbb17c77dbbe4633db4ea91fe64cc24bc5952069f140813d95940bb3a2"} Jul 17 07:12:59.510226 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:59.510136 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65fcb7c85f-d4wct" event={"ID":"9def99f9-2e41-4725-9f28-ba63f7990920","Type":"ContainerStarted","Data":"ac6712355902095e448f6c9a5e27cd5f65c654ebd24f0ae6f6c7ea12bb381bc4"} Jul 17 07:12:59.528368 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:59.528325 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-khqt9" podStartSLOduration=18.455693928 podStartE2EDuration="37.528312459s" totalImagesPullingTime="19.072618531s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 07:12:22 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:12:59.527171575 +0000 UTC m=+84.011860549" watchObservedRunningTime="2026-07-17 07:12:59.528312459 +0000 UTC m=+84.013001433" Jul 17 07:12:59.546299 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:12:59.546252 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65fcb7c85f-d4wct" podStartSLOduration=2.9988495889999998 podStartE2EDuration="8.546239341s" totalImagesPullingTime="5.547389752s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 07:12:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:12:59.545064892 +0000 UTC m=+84.029753865" watchObservedRunningTime="2026-07-17 07:12:59.546239341 +0000 UTC m=+84.030928344" Jul 17 07:13:03.488107 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:03.488076 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-vc255" Jul 17 07:13:09.462744 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:09.462717 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6675666ffd-gnf79" Jul 17 07:13:13.690043 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:13.690010 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6675666ffd-gnf79"] Jul 17 07:13:14.566812 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:14.566782 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-xr6pg_2ce00a54-15e9-42b7-b3a8-9e58d1101558/cluster-monitoring-operator/0.log" Jul 17 07:13:14.567006 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:14.566823 2568 generic.go:356] "Generic (PLEG): container finished" podID="2ce00a54-15e9-42b7-b3a8-9e58d1101558" containerID="ad0352d2020f7e512242fddac57d1a6a4befde6033afc0d60122d97b5b9271df" exitCode=2 Jul 17 07:13:14.567006 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:14.566901 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xr6pg" event={"ID":"2ce00a54-15e9-42b7-b3a8-9e58d1101558","Type":"ContainerDied","Data":"ad0352d2020f7e512242fddac57d1a6a4befde6033afc0d60122d97b5b9271df"} Jul 17 07:13:14.567221 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:14.567207 2568 scope.go:117] "RemoveContainer" containerID="ad0352d2020f7e512242fddac57d1a6a4befde6033afc0d60122d97b5b9271df" Jul 17 07:13:15.572211 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:15.572180 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-xr6pg_2ce00a54-15e9-42b7-b3a8-9e58d1101558/cluster-monitoring-operator/0.log" Jul 17 07:13:15.572614 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:15.572266 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xr6pg" event={"ID":"2ce00a54-15e9-42b7-b3a8-9e58d1101558","Type":"ContainerStarted","Data":"488fffd25cde76415b64c001289a687c514b17dc2f0d6a2d48e36d57dac41660"} Jul 17 07:13:18.337718 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:18.337684 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-g4gmw"] Jul 17 07:13:18.529264 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:18.529232 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-g4gmw" Jul 17 07:13:18.534816 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:18.532464 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 17 07:13:18.534816 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:18.533110 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-g952c\"" Jul 17 07:13:18.534816 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:18.533397 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 17 07:13:18.534816 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:18.533860 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 17 07:13:18.534816 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:18.534120 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 17 07:13:18.664098 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:18.664067 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/455b815f-ec90-4a7e-bef3-7b4e202791cb-node-exporter-wtmp\") pod \"node-exporter-g4gmw\" (UID: \"455b815f-ec90-4a7e-bef3-7b4e202791cb\") " pod="openshift-monitoring/node-exporter-g4gmw" Jul 17 07:13:18.664098 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:18.664100 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hznsp\" (UniqueName: \"kubernetes.io/projected/455b815f-ec90-4a7e-bef3-7b4e202791cb-kube-api-access-hznsp\") pod \"node-exporter-g4gmw\" (UID: \"455b815f-ec90-4a7e-bef3-7b4e202791cb\") " pod="openshift-monitoring/node-exporter-g4gmw" Jul 17 07:13:18.664353 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:18.664121 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/455b815f-ec90-4a7e-bef3-7b4e202791cb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-g4gmw\" (UID: \"455b815f-ec90-4a7e-bef3-7b4e202791cb\") " pod="openshift-monitoring/node-exporter-g4gmw" Jul 17 07:13:18.664353 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:18.664142 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/455b815f-ec90-4a7e-bef3-7b4e202791cb-metrics-client-ca\") pod \"node-exporter-g4gmw\" (UID: \"455b815f-ec90-4a7e-bef3-7b4e202791cb\") " pod="openshift-monitoring/node-exporter-g4gmw" Jul 17 07:13:18.664353 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:18.664239 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/455b815f-ec90-4a7e-bef3-7b4e202791cb-sys\") pod \"node-exporter-g4gmw\" (UID: \"455b815f-ec90-4a7e-bef3-7b4e202791cb\") " pod="openshift-monitoring/node-exporter-g4gmw" Jul 17 07:13:18.664353 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:18.664317 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/455b815f-ec90-4a7e-bef3-7b4e202791cb-node-exporter-textfile\") pod \"node-exporter-g4gmw\" (UID: \"455b815f-ec90-4a7e-bef3-7b4e202791cb\") " pod="openshift-monitoring/node-exporter-g4gmw" Jul 17 07:13:18.664485 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:18.664359 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/455b815f-ec90-4a7e-bef3-7b4e202791cb-node-exporter-accelerators-collector-config\") pod \"node-exporter-g4gmw\" (UID: \"455b815f-ec90-4a7e-bef3-7b4e202791cb\") " pod="openshift-monitoring/node-exporter-g4gmw" Jul 17 07:13:18.664485 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:18.664384 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/455b815f-ec90-4a7e-bef3-7b4e202791cb-root\") pod \"node-exporter-g4gmw\" (UID: \"455b815f-ec90-4a7e-bef3-7b4e202791cb\") " pod="openshift-monitoring/node-exporter-g4gmw" Jul 17 07:13:18.664485 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:18.664403 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/455b815f-ec90-4a7e-bef3-7b4e202791cb-node-exporter-tls\") pod \"node-exporter-g4gmw\" (UID: \"455b815f-ec90-4a7e-bef3-7b4e202791cb\") " pod="openshift-monitoring/node-exporter-g4gmw" Jul 17 07:13:18.765345 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:18.765277 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/455b815f-ec90-4a7e-bef3-7b4e202791cb-node-exporter-accelerators-collector-config\") pod \"node-exporter-g4gmw\" (UID: \"455b815f-ec90-4a7e-bef3-7b4e202791cb\") " pod="openshift-monitoring/node-exporter-g4gmw" Jul 17 07:13:18.765345 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:18.765346 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/455b815f-ec90-4a7e-bef3-7b4e202791cb-root\") pod \"node-exporter-g4gmw\" (UID: \"455b815f-ec90-4a7e-bef3-7b4e202791cb\") " pod="openshift-monitoring/node-exporter-g4gmw" Jul 17 07:13:18.765575 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:18.765367 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/455b815f-ec90-4a7e-bef3-7b4e202791cb-node-exporter-tls\") pod \"node-exporter-g4gmw\" (UID: \"455b815f-ec90-4a7e-bef3-7b4e202791cb\") " pod="openshift-monitoring/node-exporter-g4gmw" Jul 17 07:13:18.765575 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:18.765384 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/455b815f-ec90-4a7e-bef3-7b4e202791cb-node-exporter-wtmp\") pod \"node-exporter-g4gmw\" (UID: \"455b815f-ec90-4a7e-bef3-7b4e202791cb\") " pod="openshift-monitoring/node-exporter-g4gmw" Jul 17 07:13:18.765575 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:18.765399 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hznsp\" (UniqueName: \"kubernetes.io/projected/455b815f-ec90-4a7e-bef3-7b4e202791cb-kube-api-access-hznsp\") pod \"node-exporter-g4gmw\" (UID: \"455b815f-ec90-4a7e-bef3-7b4e202791cb\") " pod="openshift-monitoring/node-exporter-g4gmw" Jul 17 07:13:18.765575 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:18.765420 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/455b815f-ec90-4a7e-bef3-7b4e202791cb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-g4gmw\" (UID: \"455b815f-ec90-4a7e-bef3-7b4e202791cb\") " pod="openshift-monitoring/node-exporter-g4gmw" Jul 17 07:13:18.765575 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:18.765469 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/455b815f-ec90-4a7e-bef3-7b4e202791cb-metrics-client-ca\") pod \"node-exporter-g4gmw\" (UID: \"455b815f-ec90-4a7e-bef3-7b4e202791cb\") " pod="openshift-monitoring/node-exporter-g4gmw" Jul 17 07:13:18.765575 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:18.765506 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/455b815f-ec90-4a7e-bef3-7b4e202791cb-sys\") pod \"node-exporter-g4gmw\" (UID: \"455b815f-ec90-4a7e-bef3-7b4e202791cb\") " pod="openshift-monitoring/node-exporter-g4gmw" Jul 17 07:13:18.765575 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:18.765557 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/455b815f-ec90-4a7e-bef3-7b4e202791cb-node-exporter-textfile\") pod \"node-exporter-g4gmw\" (UID: \"455b815f-ec90-4a7e-bef3-7b4e202791cb\") " pod="openshift-monitoring/node-exporter-g4gmw" Jul 17 07:13:18.765868 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:18.765582 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/455b815f-ec90-4a7e-bef3-7b4e202791cb-node-exporter-wtmp\") pod \"node-exporter-g4gmw\" (UID: \"455b815f-ec90-4a7e-bef3-7b4e202791cb\") " pod="openshift-monitoring/node-exporter-g4gmw" Jul 17 07:13:18.765868 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:18.765567 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/455b815f-ec90-4a7e-bef3-7b4e202791cb-root\") pod \"node-exporter-g4gmw\" (UID: \"455b815f-ec90-4a7e-bef3-7b4e202791cb\") " pod="openshift-monitoring/node-exporter-g4gmw" Jul 17 07:13:18.765868 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:18.765624 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/455b815f-ec90-4a7e-bef3-7b4e202791cb-sys\") pod \"node-exporter-g4gmw\" (UID: \"455b815f-ec90-4a7e-bef3-7b4e202791cb\") " pod="openshift-monitoring/node-exporter-g4gmw" Jul 17 07:13:18.765868 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:18.765846 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/455b815f-ec90-4a7e-bef3-7b4e202791cb-node-exporter-textfile\") pod \"node-exporter-g4gmw\" (UID: \"455b815f-ec90-4a7e-bef3-7b4e202791cb\") " pod="openshift-monitoring/node-exporter-g4gmw" Jul 17 07:13:18.766088 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:18.766065 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/455b815f-ec90-4a7e-bef3-7b4e202791cb-metrics-client-ca\") pod \"node-exporter-g4gmw\" (UID: \"455b815f-ec90-4a7e-bef3-7b4e202791cb\") " pod="openshift-monitoring/node-exporter-g4gmw" Jul 17 07:13:18.766180 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:18.766154 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/455b815f-ec90-4a7e-bef3-7b4e202791cb-node-exporter-accelerators-collector-config\") pod \"node-exporter-g4gmw\" (UID: \"455b815f-ec90-4a7e-bef3-7b4e202791cb\") " pod="openshift-monitoring/node-exporter-g4gmw" Jul 17 07:13:18.767844 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:18.767821 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/455b815f-ec90-4a7e-bef3-7b4e202791cb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-g4gmw\" (UID: \"455b815f-ec90-4a7e-bef3-7b4e202791cb\") " pod="openshift-monitoring/node-exporter-g4gmw" Jul 17 07:13:18.767955 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:18.767886 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/455b815f-ec90-4a7e-bef3-7b4e202791cb-node-exporter-tls\") pod \"node-exporter-g4gmw\" (UID: \"455b815f-ec90-4a7e-bef3-7b4e202791cb\") " pod="openshift-monitoring/node-exporter-g4gmw" Jul 17 07:13:18.774900 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:18.774880 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hznsp\" (UniqueName: \"kubernetes.io/projected/455b815f-ec90-4a7e-bef3-7b4e202791cb-kube-api-access-hznsp\") pod \"node-exporter-g4gmw\" (UID: \"455b815f-ec90-4a7e-bef3-7b4e202791cb\") " pod="openshift-monitoring/node-exporter-g4gmw" Jul 17 07:13:18.841007 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:18.840975 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-g4gmw" Jul 17 07:13:18.871217 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:13:18.871186 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod455b815f_ec90_4a7e_bef3_7b4e202791cb.slice/crio-0bcdc0f3b5de601473b403cf72d9353de0b2663b31efeecb7275528bcbabb26f WatchSource:0}: Error finding container 0bcdc0f3b5de601473b403cf72d9353de0b2663b31efeecb7275528bcbabb26f: Status 404 returned error can't find the container with id 0bcdc0f3b5de601473b403cf72d9353de0b2663b31efeecb7275528bcbabb26f Jul 17 07:13:19.460604 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:19.460569 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-dtjs7" Jul 17 07:13:19.588246 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:19.588212 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-g4gmw" event={"ID":"455b815f-ec90-4a7e-bef3-7b4e202791cb","Type":"ContainerStarted","Data":"0bcdc0f3b5de601473b403cf72d9353de0b2663b31efeecb7275528bcbabb26f"} Jul 17 07:13:20.593535 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:20.593495 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-g4gmw" event={"ID":"455b815f-ec90-4a7e-bef3-7b4e202791cb","Type":"ContainerStarted","Data":"0a50f5c6d8d72fb9923d51948b2e6e03bc33f6f513efb82c1cf8ae355df3082f"} Jul 17 07:13:21.599412 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:21.599371 2568 generic.go:356] "Generic (PLEG): container finished" podID="455b815f-ec90-4a7e-bef3-7b4e202791cb" containerID="0a50f5c6d8d72fb9923d51948b2e6e03bc33f6f513efb82c1cf8ae355df3082f" exitCode=0 Jul 17 07:13:21.599797 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:21.599455 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-g4gmw" event={"ID":"455b815f-ec90-4a7e-bef3-7b4e202791cb","Type":"ContainerDied","Data":"0a50f5c6d8d72fb9923d51948b2e6e03bc33f6f513efb82c1cf8ae355df3082f"} Jul 17 07:13:22.609388 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:22.609342 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-g4gmw" event={"ID":"455b815f-ec90-4a7e-bef3-7b4e202791cb","Type":"ContainerStarted","Data":"a3b4fd44a832aee491e0b6cc834e7454019ac379ea3431a5b8d94314a2fc2046"} Jul 17 07:13:22.609758 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:22.609443 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-g4gmw" event={"ID":"455b815f-ec90-4a7e-bef3-7b4e202791cb","Type":"ContainerStarted","Data":"7fc2c973f50da88de0d4b4281bda270094f8068627ed7524bd9f8a9ba9c6f3b1"} Jul 17 07:13:22.631182 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:22.631130 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-g4gmw" podStartSLOduration=3.015372232 podStartE2EDuration="4.631117519s" totalImagesPullingTime="1.532363451s" totalInitContainerRuntime="83.381836ms" isStatefulPod=true podCreationTimestamp="2026-07-17 07:13:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:13:22.630033942 +0000 UTC m=+107.114722919" watchObservedRunningTime="2026-07-17 07:13:22.631117519 +0000 UTC m=+107.115806542" Jul 17 07:13:38.712878 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:38.712821 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6675666ffd-gnf79" podUID="0161fccb-b49c-4592-8a96-b79d237fe749" containerName="registry" containerID="cri-o://b33b9d6f46c4e625e542828f0056b377e022f551e98db0dfbeaa4af27f9429a3" gracePeriod=30 Jul 17 07:13:38.959576 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:38.959553 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6675666ffd-gnf79" Jul 17 07:13:39.018140 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:39.018054 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kndrl\" (UniqueName: \"kubernetes.io/projected/0161fccb-b49c-4592-8a96-b79d237fe749-kube-api-access-kndrl\") pod \"0161fccb-b49c-4592-8a96-b79d237fe749\" (UID: \"0161fccb-b49c-4592-8a96-b79d237fe749\") " Jul 17 07:13:39.018140 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:39.018099 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0161fccb-b49c-4592-8a96-b79d237fe749-bound-sa-token\") pod \"0161fccb-b49c-4592-8a96-b79d237fe749\" (UID: \"0161fccb-b49c-4592-8a96-b79d237fe749\") " Jul 17 07:13:39.018140 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:39.018126 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0161fccb-b49c-4592-8a96-b79d237fe749-image-registry-private-configuration\") pod \"0161fccb-b49c-4592-8a96-b79d237fe749\" (UID: \"0161fccb-b49c-4592-8a96-b79d237fe749\") " Jul 17 07:13:39.018412 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:39.018245 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0161fccb-b49c-4592-8a96-b79d237fe749-ca-trust-extracted\") pod \"0161fccb-b49c-4592-8a96-b79d237fe749\" (UID: \"0161fccb-b49c-4592-8a96-b79d237fe749\") " Jul 17 07:13:39.018412 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:39.018310 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0161fccb-b49c-4592-8a96-b79d237fe749-registry-certificates\") pod \"0161fccb-b49c-4592-8a96-b79d237fe749\" (UID: \"0161fccb-b49c-4592-8a96-b79d237fe749\") " Jul 17 07:13:39.018412 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:39.018382 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0161fccb-b49c-4592-8a96-b79d237fe749-registry-tls\") pod \"0161fccb-b49c-4592-8a96-b79d237fe749\" (UID: \"0161fccb-b49c-4592-8a96-b79d237fe749\") " Jul 17 07:13:39.018412 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:39.018409 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0161fccb-b49c-4592-8a96-b79d237fe749-trusted-ca\") pod \"0161fccb-b49c-4592-8a96-b79d237fe749\" (UID: \"0161fccb-b49c-4592-8a96-b79d237fe749\") " Jul 17 07:13:39.018618 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:39.018445 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0161fccb-b49c-4592-8a96-b79d237fe749-installation-pull-secrets\") pod \"0161fccb-b49c-4592-8a96-b79d237fe749\" (UID: \"0161fccb-b49c-4592-8a96-b79d237fe749\") " Jul 17 07:13:39.018755 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:39.018728 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0161fccb-b49c-4592-8a96-b79d237fe749-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "0161fccb-b49c-4592-8a96-b79d237fe749" (UID: "0161fccb-b49c-4592-8a96-b79d237fe749"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 07:13:39.018868 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:39.018824 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0161fccb-b49c-4592-8a96-b79d237fe749-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "0161fccb-b49c-4592-8a96-b79d237fe749" (UID: "0161fccb-b49c-4592-8a96-b79d237fe749"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 07:13:39.020657 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:39.020625 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0161fccb-b49c-4592-8a96-b79d237fe749-kube-api-access-kndrl" (OuterVolumeSpecName: "kube-api-access-kndrl") pod "0161fccb-b49c-4592-8a96-b79d237fe749" (UID: "0161fccb-b49c-4592-8a96-b79d237fe749"). InnerVolumeSpecName "kube-api-access-kndrl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 07:13:39.020788 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:39.020761 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0161fccb-b49c-4592-8a96-b79d237fe749-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "0161fccb-b49c-4592-8a96-b79d237fe749" (UID: "0161fccb-b49c-4592-8a96-b79d237fe749"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 07:13:39.020855 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:39.020822 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0161fccb-b49c-4592-8a96-b79d237fe749-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "0161fccb-b49c-4592-8a96-b79d237fe749" (UID: "0161fccb-b49c-4592-8a96-b79d237fe749"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 07:13:39.020855 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:39.020842 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0161fccb-b49c-4592-8a96-b79d237fe749-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "0161fccb-b49c-4592-8a96-b79d237fe749" (UID: "0161fccb-b49c-4592-8a96-b79d237fe749"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 07:13:39.021051 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:39.021030 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0161fccb-b49c-4592-8a96-b79d237fe749-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "0161fccb-b49c-4592-8a96-b79d237fe749" (UID: "0161fccb-b49c-4592-8a96-b79d237fe749"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 07:13:39.026771 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:39.026740 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0161fccb-b49c-4592-8a96-b79d237fe749-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "0161fccb-b49c-4592-8a96-b79d237fe749" (UID: "0161fccb-b49c-4592-8a96-b79d237fe749"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:13:39.119580 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:39.119536 2568 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0161fccb-b49c-4592-8a96-b79d237fe749-registry-tls\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:13:39.119580 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:39.119576 2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0161fccb-b49c-4592-8a96-b79d237fe749-trusted-ca\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:13:39.119580 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:39.119586 2568 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0161fccb-b49c-4592-8a96-b79d237fe749-installation-pull-secrets\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:13:39.119580 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:39.119596 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kndrl\" (UniqueName: \"kubernetes.io/projected/0161fccb-b49c-4592-8a96-b79d237fe749-kube-api-access-kndrl\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:13:39.119841 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:39.119605 2568 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0161fccb-b49c-4592-8a96-b79d237fe749-bound-sa-token\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:13:39.119841 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:39.119616 2568 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0161fccb-b49c-4592-8a96-b79d237fe749-image-registry-private-configuration\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:13:39.119841 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:39.119625 2568 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0161fccb-b49c-4592-8a96-b79d237fe749-ca-trust-extracted\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:13:39.119841 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:39.119633 2568 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0161fccb-b49c-4592-8a96-b79d237fe749-registry-certificates\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:13:39.670545 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:39.670508 2568 generic.go:356] "Generic (PLEG): container finished" podID="0161fccb-b49c-4592-8a96-b79d237fe749" containerID="b33b9d6f46c4e625e542828f0056b377e022f551e98db0dfbeaa4af27f9429a3" exitCode=0 Jul 17 07:13:39.670682 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:39.670596 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6675666ffd-gnf79" Jul 17 07:13:39.670682 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:39.670595 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6675666ffd-gnf79" event={"ID":"0161fccb-b49c-4592-8a96-b79d237fe749","Type":"ContainerDied","Data":"b33b9d6f46c4e625e542828f0056b377e022f551e98db0dfbeaa4af27f9429a3"} Jul 17 07:13:39.670772 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:39.670682 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6675666ffd-gnf79" event={"ID":"0161fccb-b49c-4592-8a96-b79d237fe749","Type":"ContainerDied","Data":"d39e23320c8138a671bbb03a947796fa9489690532c10f2de4b89861a090b66e"} Jul 17 07:13:39.670772 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:39.670705 2568 scope.go:117] "RemoveContainer" containerID="b33b9d6f46c4e625e542828f0056b377e022f551e98db0dfbeaa4af27f9429a3" Jul 17 07:13:39.678998 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:39.678977 2568 scope.go:117] "RemoveContainer" containerID="b33b9d6f46c4e625e542828f0056b377e022f551e98db0dfbeaa4af27f9429a3" Jul 17 07:13:39.679252 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:13:39.679229 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b33b9d6f46c4e625e542828f0056b377e022f551e98db0dfbeaa4af27f9429a3\": container with ID starting with b33b9d6f46c4e625e542828f0056b377e022f551e98db0dfbeaa4af27f9429a3 not found: ID does not exist" containerID="b33b9d6f46c4e625e542828f0056b377e022f551e98db0dfbeaa4af27f9429a3" Jul 17 07:13:39.679350 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:39.679259 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b33b9d6f46c4e625e542828f0056b377e022f551e98db0dfbeaa4af27f9429a3"} err="failed to get container status \"b33b9d6f46c4e625e542828f0056b377e022f551e98db0dfbeaa4af27f9429a3\": rpc error: code = NotFound desc = could not find container \"b33b9d6f46c4e625e542828f0056b377e022f551e98db0dfbeaa4af27f9429a3\": container with ID starting with b33b9d6f46c4e625e542828f0056b377e022f551e98db0dfbeaa4af27f9429a3 not found: ID does not exist" Jul 17 07:13:39.691721 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:39.691700 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6675666ffd-gnf79"] Jul 17 07:13:39.695127 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:39.695108 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6675666ffd-gnf79"] Jul 17 07:13:39.997838 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:39.997757 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0161fccb-b49c-4592-8a96-b79d237fe749" path="/var/lib/kubelet/pods/0161fccb-b49c-4592-8a96-b79d237fe749/volumes" Jul 17 07:13:48.703742 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:48.703709 2568 generic.go:356] "Generic (PLEG): container finished" podID="4c8e8c69-d8f8-4285-bed5-b527c4d915f8" containerID="d2e6ed6a3081c66fb2d3000f2f6d97a4f607c165a5d353bd2013347999d45d0b" exitCode=0 Jul 17 07:13:48.704140 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:48.703781 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-44ftt" event={"ID":"4c8e8c69-d8f8-4285-bed5-b527c4d915f8","Type":"ContainerDied","Data":"d2e6ed6a3081c66fb2d3000f2f6d97a4f607c165a5d353bd2013347999d45d0b"} Jul 17 07:13:48.704140 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:48.704084 2568 scope.go:117] "RemoveContainer" containerID="d2e6ed6a3081c66fb2d3000f2f6d97a4f607c165a5d353bd2013347999d45d0b" Jul 17 07:13:49.708931 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:49.708894 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-44ftt" event={"ID":"4c8e8c69-d8f8-4285-bed5-b527c4d915f8","Type":"ContainerStarted","Data":"c4d9f6ec0b12afcd961e6ce1074e222f669adce599375912fc9df0211739cc39"} Jul 17 07:13:57.738073 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:57.738037 2568 generic.go:356] "Generic (PLEG): container finished" podID="2a3b3827-cd5d-4292-8ce8-4e1c17c4c1e4" containerID="6175b80a3b217fe5e6cf6e78ad1e774a5ab555af362b8a92002a8b0c098c03ce" exitCode=0 Jul 17 07:13:57.738507 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:57.738112 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-jv9pk" event={"ID":"2a3b3827-cd5d-4292-8ce8-4e1c17c4c1e4","Type":"ContainerDied","Data":"6175b80a3b217fe5e6cf6e78ad1e774a5ab555af362b8a92002a8b0c098c03ce"} Jul 17 07:13:57.738507 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:57.738444 2568 scope.go:117] "RemoveContainer" containerID="6175b80a3b217fe5e6cf6e78ad1e774a5ab555af362b8a92002a8b0c098c03ce" Jul 17 07:13:58.743948 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:13:58.743918 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-jv9pk" event={"ID":"2a3b3827-cd5d-4292-8ce8-4e1c17c4c1e4","Type":"ContainerStarted","Data":"837a1541d2e897996b638dd71f0288b40020a3e56c51a7b1006f4e5368871fcd"} Jul 17 07:15:15.982553 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:15.982519 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxmkw"] Jul 17 07:15:15.983010 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:15.982793 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0161fccb-b49c-4592-8a96-b79d237fe749" containerName="registry" Jul 17 07:15:15.983010 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:15.982803 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="0161fccb-b49c-4592-8a96-b79d237fe749" containerName="registry" Jul 17 07:15:15.983010 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:15.982859 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="0161fccb-b49c-4592-8a96-b79d237fe749" containerName="registry" Jul 17 07:15:15.985905 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:15.985887 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxmkw" Jul 17 07:15:15.988991 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:15.988969 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-hnpjm\"" Jul 17 07:15:15.989100 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:15.988969 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 17 07:15:15.989100 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:15.988969 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 17 07:15:15.998825 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:15.998785 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxmkw"] Jul 17 07:15:16.013042 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:16.013013 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bpzm\" (UniqueName: \"kubernetes.io/projected/3696f519-f702-469c-8ae8-d1bb28dee9a6-kube-api-access-6bpzm\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxmkw\" (UID: \"3696f519-f702-469c-8ae8-d1bb28dee9a6\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxmkw" Jul 17 07:15:16.013169 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:16.013046 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3696f519-f702-469c-8ae8-d1bb28dee9a6-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxmkw\" (UID: \"3696f519-f702-469c-8ae8-d1bb28dee9a6\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxmkw" Jul 17 07:15:16.013169 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:16.013069 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3696f519-f702-469c-8ae8-d1bb28dee9a6-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxmkw\" (UID: \"3696f519-f702-469c-8ae8-d1bb28dee9a6\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxmkw" Jul 17 07:15:16.113955 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:16.113925 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bpzm\" (UniqueName: \"kubernetes.io/projected/3696f519-f702-469c-8ae8-d1bb28dee9a6-kube-api-access-6bpzm\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxmkw\" (UID: \"3696f519-f702-469c-8ae8-d1bb28dee9a6\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxmkw" Jul 17 07:15:16.114084 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:16.113960 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3696f519-f702-469c-8ae8-d1bb28dee9a6-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxmkw\" (UID: \"3696f519-f702-469c-8ae8-d1bb28dee9a6\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxmkw" Jul 17 07:15:16.114084 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:16.113979 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3696f519-f702-469c-8ae8-d1bb28dee9a6-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxmkw\" (UID: \"3696f519-f702-469c-8ae8-d1bb28dee9a6\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxmkw" Jul 17 07:15:16.114308 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:16.114272 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3696f519-f702-469c-8ae8-d1bb28dee9a6-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxmkw\" (UID: \"3696f519-f702-469c-8ae8-d1bb28dee9a6\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxmkw" Jul 17 07:15:16.114372 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:16.114329 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3696f519-f702-469c-8ae8-d1bb28dee9a6-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxmkw\" (UID: \"3696f519-f702-469c-8ae8-d1bb28dee9a6\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxmkw" Jul 17 07:15:16.123180 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:16.123156 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bpzm\" (UniqueName: \"kubernetes.io/projected/3696f519-f702-469c-8ae8-d1bb28dee9a6-kube-api-access-6bpzm\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxmkw\" (UID: \"3696f519-f702-469c-8ae8-d1bb28dee9a6\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxmkw" Jul 17 07:15:16.294884 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:16.294808 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxmkw" Jul 17 07:15:16.416979 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:16.416954 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxmkw"] Jul 17 07:15:16.419539 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:15:16.419501 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3696f519_f702_469c_8ae8_d1bb28dee9a6.slice/crio-6d49333e6356f9f04888b7bf06cb16d513f032207f7186663a41d9114c1e7d52 WatchSource:0}: Error finding container 6d49333e6356f9f04888b7bf06cb16d513f032207f7186663a41d9114c1e7d52: Status 404 returned error can't find the container with id 6d49333e6356f9f04888b7bf06cb16d513f032207f7186663a41d9114c1e7d52 Jul 17 07:15:17.019277 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:17.019243 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxmkw" event={"ID":"3696f519-f702-469c-8ae8-d1bb28dee9a6","Type":"ContainerStarted","Data":"6d49333e6356f9f04888b7bf06cb16d513f032207f7186663a41d9114c1e7d52"} Jul 17 07:15:24.045392 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:24.045353 2568 generic.go:356] "Generic (PLEG): container finished" podID="3696f519-f702-469c-8ae8-d1bb28dee9a6" containerID="3f71f349b697dadc01ab4e81fdc2b79bfdf73e5b5566736537b984375334e006" exitCode=0 Jul 17 07:15:24.045888 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:24.045438 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxmkw" event={"ID":"3696f519-f702-469c-8ae8-d1bb28dee9a6","Type":"ContainerDied","Data":"3f71f349b697dadc01ab4e81fdc2b79bfdf73e5b5566736537b984375334e006"} Jul 17 07:15:28.061722 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:28.061683 2568 generic.go:356] "Generic (PLEG): container finished" podID="3696f519-f702-469c-8ae8-d1bb28dee9a6" containerID="f9067f7eb59951be98ce1766766efcbbdb6a0a268eaffa0cd28491952a82d8d3" exitCode=0 Jul 17 07:15:28.062118 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:28.061774 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxmkw" event={"ID":"3696f519-f702-469c-8ae8-d1bb28dee9a6","Type":"ContainerDied","Data":"f9067f7eb59951be98ce1766766efcbbdb6a0a268eaffa0cd28491952a82d8d3"} Jul 17 07:15:35.090848 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:35.090814 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxmkw" event={"ID":"3696f519-f702-469c-8ae8-d1bb28dee9a6","Type":"ContainerStarted","Data":"520ad22cd392a6f48fe139955dbc660ffe19b44c45b47481f5ac012218831164"} Jul 17 07:15:35.114996 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:35.114946 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxmkw" podStartSLOduration=3.612879746 podStartE2EDuration="20.114928901s" totalImagesPullingTime="16.502049155s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 07:15:15 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:15:35.109510069 +0000 UTC m=+239.594199044" watchObservedRunningTime="2026-07-17 07:15:35.114928901 +0000 UTC m=+239.599617875" Jul 17 07:15:36.096198 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:36.096160 2568 generic.go:356] "Generic (PLEG): container finished" podID="3696f519-f702-469c-8ae8-d1bb28dee9a6" containerID="520ad22cd392a6f48fe139955dbc660ffe19b44c45b47481f5ac012218831164" exitCode=0 Jul 17 07:15:36.096565 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:36.096215 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxmkw" event={"ID":"3696f519-f702-469c-8ae8-d1bb28dee9a6","Type":"ContainerDied","Data":"520ad22cd392a6f48fe139955dbc660ffe19b44c45b47481f5ac012218831164"} Jul 17 07:15:37.216147 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:37.216124 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxmkw" Jul 17 07:15:37.278093 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:37.278066 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3696f519-f702-469c-8ae8-d1bb28dee9a6-util\") pod \"3696f519-f702-469c-8ae8-d1bb28dee9a6\" (UID: \"3696f519-f702-469c-8ae8-d1bb28dee9a6\") " Jul 17 07:15:37.278230 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:37.278127 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6bpzm\" (UniqueName: \"kubernetes.io/projected/3696f519-f702-469c-8ae8-d1bb28dee9a6-kube-api-access-6bpzm\") pod \"3696f519-f702-469c-8ae8-d1bb28dee9a6\" (UID: \"3696f519-f702-469c-8ae8-d1bb28dee9a6\") " Jul 17 07:15:37.278230 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:37.278167 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3696f519-f702-469c-8ae8-d1bb28dee9a6-bundle\") pod \"3696f519-f702-469c-8ae8-d1bb28dee9a6\" (UID: \"3696f519-f702-469c-8ae8-d1bb28dee9a6\") " Jul 17 07:15:37.278849 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:37.278827 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3696f519-f702-469c-8ae8-d1bb28dee9a6-bundle" (OuterVolumeSpecName: "bundle") pod "3696f519-f702-469c-8ae8-d1bb28dee9a6" (UID: "3696f519-f702-469c-8ae8-d1bb28dee9a6"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:15:37.280358 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:37.280323 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3696f519-f702-469c-8ae8-d1bb28dee9a6-kube-api-access-6bpzm" (OuterVolumeSpecName: "kube-api-access-6bpzm") pod "3696f519-f702-469c-8ae8-d1bb28dee9a6" (UID: "3696f519-f702-469c-8ae8-d1bb28dee9a6"). InnerVolumeSpecName "kube-api-access-6bpzm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 07:15:37.282485 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:37.282466 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3696f519-f702-469c-8ae8-d1bb28dee9a6-util" (OuterVolumeSpecName: "util") pod "3696f519-f702-469c-8ae8-d1bb28dee9a6" (UID: "3696f519-f702-469c-8ae8-d1bb28dee9a6"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:15:37.378980 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:37.378937 2568 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3696f519-f702-469c-8ae8-d1bb28dee9a6-bundle\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:15:37.378980 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:37.378979 2568 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3696f519-f702-469c-8ae8-d1bb28dee9a6-util\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:15:37.378980 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:37.378988 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6bpzm\" (UniqueName: \"kubernetes.io/projected/3696f519-f702-469c-8ae8-d1bb28dee9a6-kube-api-access-6bpzm\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:15:38.104588 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:38.104551 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxmkw" event={"ID":"3696f519-f702-469c-8ae8-d1bb28dee9a6","Type":"ContainerDied","Data":"6d49333e6356f9f04888b7bf06cb16d513f032207f7186663a41d9114c1e7d52"} Jul 17 07:15:38.104588 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:38.104572 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxmkw" Jul 17 07:15:38.104786 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:38.104600 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6d49333e6356f9f04888b7bf06cb16d513f032207f7186663a41d9114c1e7d52" Jul 17 07:15:44.261323 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:44.261271 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zgwtp"] Jul 17 07:15:44.261696 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:44.261546 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3696f519-f702-469c-8ae8-d1bb28dee9a6" containerName="extract" Jul 17 07:15:44.261696 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:44.261556 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="3696f519-f702-469c-8ae8-d1bb28dee9a6" containerName="extract" Jul 17 07:15:44.261696 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:44.261568 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3696f519-f702-469c-8ae8-d1bb28dee9a6" containerName="pull" Jul 17 07:15:44.261696 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:44.261574 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="3696f519-f702-469c-8ae8-d1bb28dee9a6" containerName="pull" Jul 17 07:15:44.261696 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:44.261583 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3696f519-f702-469c-8ae8-d1bb28dee9a6" containerName="util" Jul 17 07:15:44.261696 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:44.261589 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="3696f519-f702-469c-8ae8-d1bb28dee9a6" containerName="util" Jul 17 07:15:44.261696 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:44.261636 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="3696f519-f702-469c-8ae8-d1bb28dee9a6" containerName="extract" Jul 17 07:15:44.264650 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:44.264634 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zgwtp" Jul 17 07:15:44.267202 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:44.267182 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 17 07:15:44.267329 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:44.267299 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 17 07:15:44.267581 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:44.267566 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-22l24\"" Jul 17 07:15:44.278804 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:44.278781 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zgwtp"] Jul 17 07:15:44.327333 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:44.327305 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xtfg\" (UniqueName: \"kubernetes.io/projected/e0e47dd3-9c90-43f6-8e22-f3dec2ab7b42-kube-api-access-4xtfg\") pod \"cert-manager-operator-controller-manager-55c444bf86-zgwtp\" (UID: \"e0e47dd3-9c90-43f6-8e22-f3dec2ab7b42\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zgwtp" Jul 17 07:15:44.327426 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:44.327350 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e0e47dd3-9c90-43f6-8e22-f3dec2ab7b42-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-zgwtp\" (UID: \"e0e47dd3-9c90-43f6-8e22-f3dec2ab7b42\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zgwtp" Jul 17 07:15:44.428321 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:44.428274 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e0e47dd3-9c90-43f6-8e22-f3dec2ab7b42-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-zgwtp\" (UID: \"e0e47dd3-9c90-43f6-8e22-f3dec2ab7b42\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zgwtp" Jul 17 07:15:44.428466 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:44.428368 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xtfg\" (UniqueName: \"kubernetes.io/projected/e0e47dd3-9c90-43f6-8e22-f3dec2ab7b42-kube-api-access-4xtfg\") pod \"cert-manager-operator-controller-manager-55c444bf86-zgwtp\" (UID: \"e0e47dd3-9c90-43f6-8e22-f3dec2ab7b42\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zgwtp" Jul 17 07:15:44.428647 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:44.428630 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e0e47dd3-9c90-43f6-8e22-f3dec2ab7b42-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-zgwtp\" (UID: \"e0e47dd3-9c90-43f6-8e22-f3dec2ab7b42\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zgwtp" Jul 17 07:15:44.439141 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:44.439110 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xtfg\" (UniqueName: \"kubernetes.io/projected/e0e47dd3-9c90-43f6-8e22-f3dec2ab7b42-kube-api-access-4xtfg\") pod \"cert-manager-operator-controller-manager-55c444bf86-zgwtp\" (UID: \"e0e47dd3-9c90-43f6-8e22-f3dec2ab7b42\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zgwtp" Jul 17 07:15:44.574108 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:44.574012 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zgwtp" Jul 17 07:15:44.706058 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:44.706022 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zgwtp"] Jul 17 07:15:44.709982 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:15:44.709958 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0e47dd3_9c90_43f6_8e22_f3dec2ab7b42.slice/crio-afd5c373d877eeb4a972895612e318df0f14ed6de1b6041a68c05955cbca4fe9 WatchSource:0}: Error finding container afd5c373d877eeb4a972895612e318df0f14ed6de1b6041a68c05955cbca4fe9: Status 404 returned error can't find the container with id afd5c373d877eeb4a972895612e318df0f14ed6de1b6041a68c05955cbca4fe9 Jul 17 07:15:45.130139 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:45.130101 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zgwtp" event={"ID":"e0e47dd3-9c90-43f6-8e22-f3dec2ab7b42","Type":"ContainerStarted","Data":"afd5c373d877eeb4a972895612e318df0f14ed6de1b6041a68c05955cbca4fe9"} Jul 17 07:15:47.142948 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:47.142913 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zgwtp" event={"ID":"e0e47dd3-9c90-43f6-8e22-f3dec2ab7b42","Type":"ContainerStarted","Data":"a3355427bb5083866b49beb9635c5df0dde2d5e19e28d90b69c5e01d4651a48c"} Jul 17 07:15:47.202806 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:47.202755 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zgwtp" podStartSLOduration=1.77389608 podStartE2EDuration="3.202741314s" totalImagesPullingTime="1.428845234s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 07:15:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:15:47.201647243 +0000 UTC m=+251.686336229" watchObservedRunningTime="2026-07-17 07:15:47.202741314 +0000 UTC m=+251.687430287" Jul 17 07:15:47.742083 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:47.742045 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz8nww"] Jul 17 07:15:47.745540 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:47.745519 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz8nww" Jul 17 07:15:47.748170 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:47.748147 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-hnpjm\"" Jul 17 07:15:47.748331 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:47.748311 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 17 07:15:47.749006 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:47.748989 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 17 07:15:47.757902 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:47.757880 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz8nww"] Jul 17 07:15:47.855749 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:47.855715 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4631fdf6-a356-41b6-b57e-5a35824fa4e2-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz8nww\" (UID: \"4631fdf6-a356-41b6-b57e-5a35824fa4e2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz8nww" Jul 17 07:15:47.855880 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:47.855754 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftnkz\" (UniqueName: \"kubernetes.io/projected/4631fdf6-a356-41b6-b57e-5a35824fa4e2-kube-api-access-ftnkz\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz8nww\" (UID: \"4631fdf6-a356-41b6-b57e-5a35824fa4e2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz8nww" Jul 17 07:15:47.855880 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:47.855821 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4631fdf6-a356-41b6-b57e-5a35824fa4e2-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz8nww\" (UID: \"4631fdf6-a356-41b6-b57e-5a35824fa4e2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz8nww" Jul 17 07:15:47.957175 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:47.957132 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4631fdf6-a356-41b6-b57e-5a35824fa4e2-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz8nww\" (UID: \"4631fdf6-a356-41b6-b57e-5a35824fa4e2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz8nww" Jul 17 07:15:47.957391 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:47.957199 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftnkz\" (UniqueName: \"kubernetes.io/projected/4631fdf6-a356-41b6-b57e-5a35824fa4e2-kube-api-access-ftnkz\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz8nww\" (UID: \"4631fdf6-a356-41b6-b57e-5a35824fa4e2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz8nww" Jul 17 07:15:47.957391 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:47.957261 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4631fdf6-a356-41b6-b57e-5a35824fa4e2-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz8nww\" (UID: \"4631fdf6-a356-41b6-b57e-5a35824fa4e2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz8nww" Jul 17 07:15:47.957664 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:47.957642 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4631fdf6-a356-41b6-b57e-5a35824fa4e2-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz8nww\" (UID: \"4631fdf6-a356-41b6-b57e-5a35824fa4e2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz8nww" Jul 17 07:15:47.957768 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:47.957648 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4631fdf6-a356-41b6-b57e-5a35824fa4e2-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz8nww\" (UID: \"4631fdf6-a356-41b6-b57e-5a35824fa4e2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz8nww" Jul 17 07:15:47.992486 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:47.992422 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftnkz\" (UniqueName: \"kubernetes.io/projected/4631fdf6-a356-41b6-b57e-5a35824fa4e2-kube-api-access-ftnkz\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz8nww\" (UID: \"4631fdf6-a356-41b6-b57e-5a35824fa4e2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz8nww" Jul 17 07:15:48.055104 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:48.055079 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz8nww" Jul 17 07:15:48.191748 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:48.191723 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz8nww"] Jul 17 07:15:48.194367 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:15:48.194341 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4631fdf6_a356_41b6_b57e_5a35824fa4e2.slice/crio-8bd04e46af63146f576abf0b6b5675ae59c2e0e9698ff9e9c49b1a2772f6c03c WatchSource:0}: Error finding container 8bd04e46af63146f576abf0b6b5675ae59c2e0e9698ff9e9c49b1a2772f6c03c: Status 404 returned error can't find the container with id 8bd04e46af63146f576abf0b6b5675ae59c2e0e9698ff9e9c49b1a2772f6c03c Jul 17 07:15:49.152865 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:49.152828 2568 generic.go:356] "Generic (PLEG): container finished" podID="4631fdf6-a356-41b6-b57e-5a35824fa4e2" containerID="4b0429c3d2c79846c4a431a69f3986bfa0c6f91381c1e0bbe210859c02ae84e1" exitCode=0 Jul 17 07:15:49.153040 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:49.152923 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz8nww" event={"ID":"4631fdf6-a356-41b6-b57e-5a35824fa4e2","Type":"ContainerDied","Data":"4b0429c3d2c79846c4a431a69f3986bfa0c6f91381c1e0bbe210859c02ae84e1"} Jul 17 07:15:49.153040 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:49.152976 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz8nww" event={"ID":"4631fdf6-a356-41b6-b57e-5a35824fa4e2","Type":"ContainerStarted","Data":"8bd04e46af63146f576abf0b6b5675ae59c2e0e9698ff9e9c49b1a2772f6c03c"} Jul 17 07:15:51.164763 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:51.164735 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz8nww" event={"ID":"4631fdf6-a356-41b6-b57e-5a35824fa4e2","Type":"ContainerStarted","Data":"217656c252d5af5cb0d4d06ea03ddac9e97e2fcf4ea0be985f85e00030d92756"} Jul 17 07:15:52.170814 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:52.170776 2568 generic.go:356] "Generic (PLEG): container finished" podID="4631fdf6-a356-41b6-b57e-5a35824fa4e2" containerID="217656c252d5af5cb0d4d06ea03ddac9e97e2fcf4ea0be985f85e00030d92756" exitCode=0 Jul 17 07:15:52.171202 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:52.170823 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz8nww" event={"ID":"4631fdf6-a356-41b6-b57e-5a35824fa4e2","Type":"ContainerDied","Data":"217656c252d5af5cb0d4d06ea03ddac9e97e2fcf4ea0be985f85e00030d92756"} Jul 17 07:15:53.177506 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:53.177468 2568 generic.go:356] "Generic (PLEG): container finished" podID="4631fdf6-a356-41b6-b57e-5a35824fa4e2" containerID="5ef4cc640a158537ed4ae402066d4d03c071f551903dce705b81b225ecf70a01" exitCode=0 Jul 17 07:15:53.177884 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:53.177547 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz8nww" event={"ID":"4631fdf6-a356-41b6-b57e-5a35824fa4e2","Type":"ContainerDied","Data":"5ef4cc640a158537ed4ae402066d4d03c071f551903dce705b81b225ecf70a01"} Jul 17 07:15:54.319104 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:54.319075 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz8nww" Jul 17 07:15:54.409839 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:54.409811 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4631fdf6-a356-41b6-b57e-5a35824fa4e2-bundle\") pod \"4631fdf6-a356-41b6-b57e-5a35824fa4e2\" (UID: \"4631fdf6-a356-41b6-b57e-5a35824fa4e2\") " Jul 17 07:15:54.409992 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:54.409852 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4631fdf6-a356-41b6-b57e-5a35824fa4e2-util\") pod \"4631fdf6-a356-41b6-b57e-5a35824fa4e2\" (UID: \"4631fdf6-a356-41b6-b57e-5a35824fa4e2\") " Jul 17 07:15:54.409992 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:54.409886 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ftnkz\" (UniqueName: \"kubernetes.io/projected/4631fdf6-a356-41b6-b57e-5a35824fa4e2-kube-api-access-ftnkz\") pod \"4631fdf6-a356-41b6-b57e-5a35824fa4e2\" (UID: \"4631fdf6-a356-41b6-b57e-5a35824fa4e2\") " Jul 17 07:15:54.410255 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:54.410218 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4631fdf6-a356-41b6-b57e-5a35824fa4e2-bundle" (OuterVolumeSpecName: "bundle") pod "4631fdf6-a356-41b6-b57e-5a35824fa4e2" (UID: "4631fdf6-a356-41b6-b57e-5a35824fa4e2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:15:54.411986 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:54.411967 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4631fdf6-a356-41b6-b57e-5a35824fa4e2-kube-api-access-ftnkz" (OuterVolumeSpecName: "kube-api-access-ftnkz") pod "4631fdf6-a356-41b6-b57e-5a35824fa4e2" (UID: "4631fdf6-a356-41b6-b57e-5a35824fa4e2"). InnerVolumeSpecName "kube-api-access-ftnkz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 07:15:54.414914 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:54.414881 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4631fdf6-a356-41b6-b57e-5a35824fa4e2-util" (OuterVolumeSpecName: "util") pod "4631fdf6-a356-41b6-b57e-5a35824fa4e2" (UID: "4631fdf6-a356-41b6-b57e-5a35824fa4e2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:15:54.510911 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:54.510845 2568 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4631fdf6-a356-41b6-b57e-5a35824fa4e2-util\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:15:54.510911 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:54.510870 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ftnkz\" (UniqueName: \"kubernetes.io/projected/4631fdf6-a356-41b6-b57e-5a35824fa4e2-kube-api-access-ftnkz\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:15:54.510911 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:54.510880 2568 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4631fdf6-a356-41b6-b57e-5a35824fa4e2-bundle\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:15:55.186926 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:55.186897 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz8nww" Jul 17 07:15:55.187164 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:55.186901 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz8nww" event={"ID":"4631fdf6-a356-41b6-b57e-5a35824fa4e2","Type":"ContainerDied","Data":"8bd04e46af63146f576abf0b6b5675ae59c2e0e9698ff9e9c49b1a2772f6c03c"} Jul 17 07:15:55.187164 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:15:55.187015 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8bd04e46af63146f576abf0b6b5675ae59c2e0e9698ff9e9c49b1a2772f6c03c" Jul 17 07:16:08.743961 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:08.743881 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdtxbh"] Jul 17 07:16:08.744326 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:08.744122 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4631fdf6-a356-41b6-b57e-5a35824fa4e2" containerName="util" Jul 17 07:16:08.744326 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:08.744132 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4631fdf6-a356-41b6-b57e-5a35824fa4e2" containerName="util" Jul 17 07:16:08.744326 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:08.744143 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4631fdf6-a356-41b6-b57e-5a35824fa4e2" containerName="extract" Jul 17 07:16:08.744326 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:08.744148 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4631fdf6-a356-41b6-b57e-5a35824fa4e2" containerName="extract" Jul 17 07:16:08.744326 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:08.744157 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4631fdf6-a356-41b6-b57e-5a35824fa4e2" containerName="pull" Jul 17 07:16:08.744326 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:08.744162 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4631fdf6-a356-41b6-b57e-5a35824fa4e2" containerName="pull" Jul 17 07:16:08.744326 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:08.744217 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="4631fdf6-a356-41b6-b57e-5a35824fa4e2" containerName="extract" Jul 17 07:16:08.800269 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:08.800239 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdtxbh"] Jul 17 07:16:08.800437 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:08.800406 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdtxbh" Jul 17 07:16:08.802978 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:08.802953 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-hnpjm\"" Jul 17 07:16:08.802978 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:08.802963 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 17 07:16:08.803187 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:08.802969 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 17 07:16:08.921405 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:08.921374 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/046cd94a-3dcb-4373-a0c1-f4998b85f2ae-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdtxbh\" (UID: \"046cd94a-3dcb-4373-a0c1-f4998b85f2ae\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdtxbh" Jul 17 07:16:08.921556 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:08.921414 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/046cd94a-3dcb-4373-a0c1-f4998b85f2ae-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdtxbh\" (UID: \"046cd94a-3dcb-4373-a0c1-f4998b85f2ae\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdtxbh" Jul 17 07:16:08.921556 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:08.921438 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z49dx\" (UniqueName: \"kubernetes.io/projected/046cd94a-3dcb-4373-a0c1-f4998b85f2ae-kube-api-access-z49dx\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdtxbh\" (UID: \"046cd94a-3dcb-4373-a0c1-f4998b85f2ae\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdtxbh" Jul 17 07:16:09.022538 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:09.022454 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/046cd94a-3dcb-4373-a0c1-f4998b85f2ae-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdtxbh\" (UID: \"046cd94a-3dcb-4373-a0c1-f4998b85f2ae\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdtxbh" Jul 17 07:16:09.022538 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:09.022495 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/046cd94a-3dcb-4373-a0c1-f4998b85f2ae-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdtxbh\" (UID: \"046cd94a-3dcb-4373-a0c1-f4998b85f2ae\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdtxbh" Jul 17 07:16:09.022538 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:09.022533 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z49dx\" (UniqueName: \"kubernetes.io/projected/046cd94a-3dcb-4373-a0c1-f4998b85f2ae-kube-api-access-z49dx\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdtxbh\" (UID: \"046cd94a-3dcb-4373-a0c1-f4998b85f2ae\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdtxbh" Jul 17 07:16:09.022832 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:09.022811 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/046cd94a-3dcb-4373-a0c1-f4998b85f2ae-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdtxbh\" (UID: \"046cd94a-3dcb-4373-a0c1-f4998b85f2ae\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdtxbh" Jul 17 07:16:09.022910 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:09.022891 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/046cd94a-3dcb-4373-a0c1-f4998b85f2ae-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdtxbh\" (UID: \"046cd94a-3dcb-4373-a0c1-f4998b85f2ae\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdtxbh" Jul 17 07:16:09.030984 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:09.030956 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z49dx\" (UniqueName: \"kubernetes.io/projected/046cd94a-3dcb-4373-a0c1-f4998b85f2ae-kube-api-access-z49dx\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdtxbh\" (UID: \"046cd94a-3dcb-4373-a0c1-f4998b85f2ae\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdtxbh" Jul 17 07:16:09.109384 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:09.109354 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdtxbh" Jul 17 07:16:09.258614 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:09.258521 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdtxbh"] Jul 17 07:16:09.261894 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:16:09.261867 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod046cd94a_3dcb_4373_a0c1_f4998b85f2ae.slice/crio-271b8107acfc1cf7161ee097ed4764ec348c871242341aaf1ad428de4d1d41bd WatchSource:0}: Error finding container 271b8107acfc1cf7161ee097ed4764ec348c871242341aaf1ad428de4d1d41bd: Status 404 returned error can't find the container with id 271b8107acfc1cf7161ee097ed4764ec348c871242341aaf1ad428de4d1d41bd Jul 17 07:16:10.248864 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:10.248829 2568 generic.go:356] "Generic (PLEG): container finished" podID="046cd94a-3dcb-4373-a0c1-f4998b85f2ae" containerID="e87e40489efd5a613e87cd9b43e42fe3fc0ec28471b4146df28ca4feb5a9d681" exitCode=0 Jul 17 07:16:10.249327 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:10.248908 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdtxbh" event={"ID":"046cd94a-3dcb-4373-a0c1-f4998b85f2ae","Type":"ContainerDied","Data":"e87e40489efd5a613e87cd9b43e42fe3fc0ec28471b4146df28ca4feb5a9d681"} Jul 17 07:16:10.249327 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:10.248984 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdtxbh" event={"ID":"046cd94a-3dcb-4373-a0c1-f4998b85f2ae","Type":"ContainerStarted","Data":"271b8107acfc1cf7161ee097ed4764ec348c871242341aaf1ad428de4d1d41bd"} Jul 17 07:16:11.254610 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:11.254580 2568 generic.go:356] "Generic (PLEG): container finished" podID="046cd94a-3dcb-4373-a0c1-f4998b85f2ae" containerID="17141de9799170ffa23f59e7ae1dbc4799f8b17fc0ec94948bff1cb1e2cbc320" exitCode=0 Jul 17 07:16:11.255004 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:11.254671 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdtxbh" event={"ID":"046cd94a-3dcb-4373-a0c1-f4998b85f2ae","Type":"ContainerDied","Data":"17141de9799170ffa23f59e7ae1dbc4799f8b17fc0ec94948bff1cb1e2cbc320"} Jul 17 07:16:12.260040 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:12.260007 2568 generic.go:356] "Generic (PLEG): container finished" podID="046cd94a-3dcb-4373-a0c1-f4998b85f2ae" containerID="e87acbf403fd16da85a0fd06331547e62a6376af736be7062388dadf57e442b5" exitCode=0 Jul 17 07:16:12.260420 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:12.260097 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdtxbh" event={"ID":"046cd94a-3dcb-4373-a0c1-f4998b85f2ae","Type":"ContainerDied","Data":"e87acbf403fd16da85a0fd06331547e62a6376af736be7062388dadf57e442b5"} Jul 17 07:16:13.386237 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:13.386215 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdtxbh" Jul 17 07:16:13.461301 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:13.461253 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/046cd94a-3dcb-4373-a0c1-f4998b85f2ae-bundle\") pod \"046cd94a-3dcb-4373-a0c1-f4998b85f2ae\" (UID: \"046cd94a-3dcb-4373-a0c1-f4998b85f2ae\") " Jul 17 07:16:13.461449 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:13.461337 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z49dx\" (UniqueName: \"kubernetes.io/projected/046cd94a-3dcb-4373-a0c1-f4998b85f2ae-kube-api-access-z49dx\") pod \"046cd94a-3dcb-4373-a0c1-f4998b85f2ae\" (UID: \"046cd94a-3dcb-4373-a0c1-f4998b85f2ae\") " Jul 17 07:16:13.461449 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:13.461365 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/046cd94a-3dcb-4373-a0c1-f4998b85f2ae-util\") pod \"046cd94a-3dcb-4373-a0c1-f4998b85f2ae\" (UID: \"046cd94a-3dcb-4373-a0c1-f4998b85f2ae\") " Jul 17 07:16:13.462021 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:13.461982 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/046cd94a-3dcb-4373-a0c1-f4998b85f2ae-bundle" (OuterVolumeSpecName: "bundle") pod "046cd94a-3dcb-4373-a0c1-f4998b85f2ae" (UID: "046cd94a-3dcb-4373-a0c1-f4998b85f2ae"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:16:13.463393 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:13.463366 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/046cd94a-3dcb-4373-a0c1-f4998b85f2ae-kube-api-access-z49dx" (OuterVolumeSpecName: "kube-api-access-z49dx") pod "046cd94a-3dcb-4373-a0c1-f4998b85f2ae" (UID: "046cd94a-3dcb-4373-a0c1-f4998b85f2ae"). InnerVolumeSpecName "kube-api-access-z49dx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 07:16:13.467068 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:13.467033 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/046cd94a-3dcb-4373-a0c1-f4998b85f2ae-util" (OuterVolumeSpecName: "util") pod "046cd94a-3dcb-4373-a0c1-f4998b85f2ae" (UID: "046cd94a-3dcb-4373-a0c1-f4998b85f2ae"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:16:13.562613 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:13.562541 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z49dx\" (UniqueName: \"kubernetes.io/projected/046cd94a-3dcb-4373-a0c1-f4998b85f2ae-kube-api-access-z49dx\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:16:13.562613 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:13.562565 2568 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/046cd94a-3dcb-4373-a0c1-f4998b85f2ae-util\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:16:13.562613 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:13.562574 2568 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/046cd94a-3dcb-4373-a0c1-f4998b85f2ae-bundle\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:16:14.270012 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:14.269930 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdtxbh" Jul 17 07:16:14.270012 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:14.269949 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdtxbh" event={"ID":"046cd94a-3dcb-4373-a0c1-f4998b85f2ae","Type":"ContainerDied","Data":"271b8107acfc1cf7161ee097ed4764ec348c871242341aaf1ad428de4d1d41bd"} Jul 17 07:16:14.270012 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:14.269991 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="271b8107acfc1cf7161ee097ed4764ec348c871242341aaf1ad428de4d1d41bd" Jul 17 07:16:22.110189 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:22.110155 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-59ffcf4dc8-56dhq"] Jul 17 07:16:22.110627 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:22.110421 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="046cd94a-3dcb-4373-a0c1-f4998b85f2ae" containerName="util" Jul 17 07:16:22.110627 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:22.110431 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="046cd94a-3dcb-4373-a0c1-f4998b85f2ae" containerName="util" Jul 17 07:16:22.110627 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:22.110443 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="046cd94a-3dcb-4373-a0c1-f4998b85f2ae" containerName="pull" Jul 17 07:16:22.110627 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:22.110449 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="046cd94a-3dcb-4373-a0c1-f4998b85f2ae" containerName="pull" Jul 17 07:16:22.110627 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:22.110462 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="046cd94a-3dcb-4373-a0c1-f4998b85f2ae" containerName="extract" Jul 17 07:16:22.110627 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:22.110467 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="046cd94a-3dcb-4373-a0c1-f4998b85f2ae" containerName="extract" Jul 17 07:16:22.110627 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:22.110511 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="046cd94a-3dcb-4373-a0c1-f4998b85f2ae" containerName="extract" Jul 17 07:16:22.116709 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:22.116691 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-59ffcf4dc8-56dhq" Jul 17 07:16:22.119634 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:22.119609 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 17 07:16:22.120515 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:22.120497 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 17 07:16:22.120617 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:22.120560 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-cjtgt\"" Jul 17 07:16:22.120617 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:22.120586 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 17 07:16:22.120720 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:22.120683 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 17 07:16:22.225359 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:22.225326 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6a4b5d36-d807-4d62-8aaf-6432767c269c-manifests\") pod \"opendatahub-operator-controller-manager-59ffcf4dc8-56dhq\" (UID: \"6a4b5d36-d807-4d62-8aaf-6432767c269c\") " pod="opendatahub/opendatahub-operator-controller-manager-59ffcf4dc8-56dhq" Jul 17 07:16:22.225507 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:22.225379 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lv6dm\" (UniqueName: \"kubernetes.io/projected/6a4b5d36-d807-4d62-8aaf-6432767c269c-kube-api-access-lv6dm\") pod \"opendatahub-operator-controller-manager-59ffcf4dc8-56dhq\" (UID: \"6a4b5d36-d807-4d62-8aaf-6432767c269c\") " pod="opendatahub/opendatahub-operator-controller-manager-59ffcf4dc8-56dhq" Jul 17 07:16:22.225507 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:22.225451 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6a4b5d36-d807-4d62-8aaf-6432767c269c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-59ffcf4dc8-56dhq\" (UID: \"6a4b5d36-d807-4d62-8aaf-6432767c269c\") " pod="opendatahub/opendatahub-operator-controller-manager-59ffcf4dc8-56dhq" Jul 17 07:16:22.225507 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:22.225482 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6a4b5d36-d807-4d62-8aaf-6432767c269c-tmp\") pod \"opendatahub-operator-controller-manager-59ffcf4dc8-56dhq\" (UID: \"6a4b5d36-d807-4d62-8aaf-6432767c269c\") " pod="opendatahub/opendatahub-operator-controller-manager-59ffcf4dc8-56dhq" Jul 17 07:16:22.225507 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:22.225505 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6a4b5d36-d807-4d62-8aaf-6432767c269c-webhook-cert\") pod \"opendatahub-operator-controller-manager-59ffcf4dc8-56dhq\" (UID: \"6a4b5d36-d807-4d62-8aaf-6432767c269c\") " pod="opendatahub/opendatahub-operator-controller-manager-59ffcf4dc8-56dhq" Jul 17 07:16:22.326554 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:22.326521 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lv6dm\" (UniqueName: \"kubernetes.io/projected/6a4b5d36-d807-4d62-8aaf-6432767c269c-kube-api-access-lv6dm\") pod \"opendatahub-operator-controller-manager-59ffcf4dc8-56dhq\" (UID: \"6a4b5d36-d807-4d62-8aaf-6432767c269c\") " pod="opendatahub/opendatahub-operator-controller-manager-59ffcf4dc8-56dhq" Jul 17 07:16:22.326695 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:22.326576 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6a4b5d36-d807-4d62-8aaf-6432767c269c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-59ffcf4dc8-56dhq\" (UID: \"6a4b5d36-d807-4d62-8aaf-6432767c269c\") " pod="opendatahub/opendatahub-operator-controller-manager-59ffcf4dc8-56dhq" Jul 17 07:16:22.326695 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:22.326597 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6a4b5d36-d807-4d62-8aaf-6432767c269c-tmp\") pod \"opendatahub-operator-controller-manager-59ffcf4dc8-56dhq\" (UID: \"6a4b5d36-d807-4d62-8aaf-6432767c269c\") " pod="opendatahub/opendatahub-operator-controller-manager-59ffcf4dc8-56dhq" Jul 17 07:16:22.326695 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:22.326618 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6a4b5d36-d807-4d62-8aaf-6432767c269c-webhook-cert\") pod \"opendatahub-operator-controller-manager-59ffcf4dc8-56dhq\" (UID: \"6a4b5d36-d807-4d62-8aaf-6432767c269c\") " pod="opendatahub/opendatahub-operator-controller-manager-59ffcf4dc8-56dhq" Jul 17 07:16:22.326695 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:22.326657 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6a4b5d36-d807-4d62-8aaf-6432767c269c-manifests\") pod \"opendatahub-operator-controller-manager-59ffcf4dc8-56dhq\" (UID: \"6a4b5d36-d807-4d62-8aaf-6432767c269c\") " pod="opendatahub/opendatahub-operator-controller-manager-59ffcf4dc8-56dhq" Jul 17 07:16:22.327152 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:22.327125 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6a4b5d36-d807-4d62-8aaf-6432767c269c-manifests\") pod \"opendatahub-operator-controller-manager-59ffcf4dc8-56dhq\" (UID: \"6a4b5d36-d807-4d62-8aaf-6432767c269c\") " pod="opendatahub/opendatahub-operator-controller-manager-59ffcf4dc8-56dhq" Jul 17 07:16:22.327401 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:22.327376 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-59ffcf4dc8-56dhq"] Jul 17 07:16:22.328943 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:22.328923 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6a4b5d36-d807-4d62-8aaf-6432767c269c-tmp\") pod \"opendatahub-operator-controller-manager-59ffcf4dc8-56dhq\" (UID: \"6a4b5d36-d807-4d62-8aaf-6432767c269c\") " pod="opendatahub/opendatahub-operator-controller-manager-59ffcf4dc8-56dhq" Jul 17 07:16:22.329044 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:22.328996 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6a4b5d36-d807-4d62-8aaf-6432767c269c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-59ffcf4dc8-56dhq\" (UID: \"6a4b5d36-d807-4d62-8aaf-6432767c269c\") " pod="opendatahub/opendatahub-operator-controller-manager-59ffcf4dc8-56dhq" Jul 17 07:16:22.329177 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:22.329160 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6a4b5d36-d807-4d62-8aaf-6432767c269c-webhook-cert\") pod \"opendatahub-operator-controller-manager-59ffcf4dc8-56dhq\" (UID: \"6a4b5d36-d807-4d62-8aaf-6432767c269c\") " pod="opendatahub/opendatahub-operator-controller-manager-59ffcf4dc8-56dhq" Jul 17 07:16:22.342635 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:22.342614 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lv6dm\" (UniqueName: \"kubernetes.io/projected/6a4b5d36-d807-4d62-8aaf-6432767c269c-kube-api-access-lv6dm\") pod \"opendatahub-operator-controller-manager-59ffcf4dc8-56dhq\" (UID: \"6a4b5d36-d807-4d62-8aaf-6432767c269c\") " pod="opendatahub/opendatahub-operator-controller-manager-59ffcf4dc8-56dhq" Jul 17 07:16:22.426187 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:22.426115 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-59ffcf4dc8-56dhq" Jul 17 07:16:22.566054 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:22.566016 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-59ffcf4dc8-56dhq"] Jul 17 07:16:22.570308 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:16:22.570257 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a4b5d36_d807_4d62_8aaf_6432767c269c.slice/crio-1047d423849dd54295616af85a65e3e6bf4f202a928e409ef9b14d89f8c796c7 WatchSource:0}: Error finding container 1047d423849dd54295616af85a65e3e6bf4f202a928e409ef9b14d89f8c796c7: Status 404 returned error can't find the container with id 1047d423849dd54295616af85a65e3e6bf4f202a928e409ef9b14d89f8c796c7 Jul 17 07:16:23.308085 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:23.308033 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59ffcf4dc8-56dhq" event={"ID":"6a4b5d36-d807-4d62-8aaf-6432767c269c","Type":"ContainerStarted","Data":"1047d423849dd54295616af85a65e3e6bf4f202a928e409ef9b14d89f8c796c7"} Jul 17 07:16:26.323415 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:26.323374 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59ffcf4dc8-56dhq" event={"ID":"6a4b5d36-d807-4d62-8aaf-6432767c269c","Type":"ContainerStarted","Data":"b5269f54d78db5bb043beac36bef481157a4c9d16ef876f85d82ed7f4b517ecb"} Jul 17 07:16:27.329045 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:27.328951 2568 generic.go:356] "Generic (PLEG): container finished" podID="6a4b5d36-d807-4d62-8aaf-6432767c269c" containerID="b5269f54d78db5bb043beac36bef481157a4c9d16ef876f85d82ed7f4b517ecb" exitCode=0 Jul 17 07:16:27.329045 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:27.329018 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59ffcf4dc8-56dhq" event={"ID":"6a4b5d36-d807-4d62-8aaf-6432767c269c","Type":"ContainerDied","Data":"b5269f54d78db5bb043beac36bef481157a4c9d16ef876f85d82ed7f4b517ecb"} Jul 17 07:16:28.334695 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:28.334657 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59ffcf4dc8-56dhq" event={"ID":"6a4b5d36-d807-4d62-8aaf-6432767c269c","Type":"ContainerStarted","Data":"d5796003b8051b4ea289aee034128e60bbaa26150fe3342b4ab0a3b6d0a9e63c"} Jul 17 07:16:28.335167 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:28.334771 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-59ffcf4dc8-56dhq" Jul 17 07:16:28.360275 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:28.360225 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-59ffcf4dc8-56dhq" podStartSLOduration=1.617384535 podStartE2EDuration="6.360210731s" totalImagesPullingTime="3.654020066s" totalInitContainerRuntime="1.08880613s" isStatefulPod=false podCreationTimestamp="2026-07-17 07:16:22 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:16:28.357714095 +0000 UTC m=+292.842403080" watchObservedRunningTime="2026-07-17 07:16:28.360210731 +0000 UTC m=+292.844899704" Jul 17 07:16:35.931500 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:35.931471 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-xr6pg_2ce00a54-15e9-42b7-b3a8-9e58d1101558/cluster-monitoring-operator/0.log" Jul 17 07:16:35.936830 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:35.936810 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-xpxv5_27538d1e-992b-4651-a2d2-7a7ecf442806/console-operator/1.log" Jul 17 07:16:35.938188 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:35.938167 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-xr6pg_2ce00a54-15e9-42b7-b3a8-9e58d1101558/cluster-monitoring-operator/0.log" Jul 17 07:16:35.940946 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:35.940928 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cpwz4_1c44a3ae-4c83-4203-8b0b-116d6618accf/ovn-acl-logging/0.log" Jul 17 07:16:35.944540 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:35.944521 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-xpxv5_27538d1e-992b-4651-a2d2-7a7ecf442806/console-operator/1.log" Jul 17 07:16:35.948371 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:35.948354 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cpwz4_1c44a3ae-4c83-4203-8b0b-116d6618accf/ovn-acl-logging/0.log" Jul 17 07:16:35.948516 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:35.948499 2568 kubelet.go:1648] "Image garbage collection succeeded" Jul 17 07:16:39.340307 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:39.340260 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-59ffcf4dc8-56dhq" Jul 17 07:16:42.652388 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:42.652344 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-7kqk8"] Jul 17 07:16:42.657265 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:42.657241 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-7kqk8" Jul 17 07:16:42.659822 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:42.659801 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 17 07:16:42.660460 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:42.660442 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-jwfbr\"" Jul 17 07:16:42.694440 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:42.694411 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-7kqk8"] Jul 17 07:16:42.792615 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:42.792582 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjznb\" (UniqueName: \"kubernetes.io/projected/8dc175df-3be1-4f44-aaed-fe6ee8e60771-kube-api-access-gjznb\") pod \"odh-model-controller-7998c67dc4-7kqk8\" (UID: \"8dc175df-3be1-4f44-aaed-fe6ee8e60771\") " pod="opendatahub/odh-model-controller-7998c67dc4-7kqk8" Jul 17 07:16:42.792771 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:42.792636 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8dc175df-3be1-4f44-aaed-fe6ee8e60771-cert\") pod \"odh-model-controller-7998c67dc4-7kqk8\" (UID: \"8dc175df-3be1-4f44-aaed-fe6ee8e60771\") " pod="opendatahub/odh-model-controller-7998c67dc4-7kqk8" Jul 17 07:16:42.893554 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:42.893520 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8dc175df-3be1-4f44-aaed-fe6ee8e60771-cert\") pod \"odh-model-controller-7998c67dc4-7kqk8\" (UID: \"8dc175df-3be1-4f44-aaed-fe6ee8e60771\") " pod="opendatahub/odh-model-controller-7998c67dc4-7kqk8" Jul 17 07:16:42.893756 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:42.893607 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjznb\" (UniqueName: \"kubernetes.io/projected/8dc175df-3be1-4f44-aaed-fe6ee8e60771-kube-api-access-gjznb\") pod \"odh-model-controller-7998c67dc4-7kqk8\" (UID: \"8dc175df-3be1-4f44-aaed-fe6ee8e60771\") " pod="opendatahub/odh-model-controller-7998c67dc4-7kqk8" Jul 17 07:16:42.893756 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:16:42.893718 2568 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 17 07:16:42.893891 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:16:42.893805 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dc175df-3be1-4f44-aaed-fe6ee8e60771-cert podName:8dc175df-3be1-4f44-aaed-fe6ee8e60771 nodeName:}" failed. No retries permitted until 2026-07-17 07:16:43.393781755 +0000 UTC m=+307.878470727 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/8dc175df-3be1-4f44-aaed-fe6ee8e60771-cert") pod "odh-model-controller-7998c67dc4-7kqk8" (UID: "8dc175df-3be1-4f44-aaed-fe6ee8e60771") : secret "odh-model-controller-webhook-cert" not found Jul 17 07:16:42.903619 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:42.903547 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjznb\" (UniqueName: \"kubernetes.io/projected/8dc175df-3be1-4f44-aaed-fe6ee8e60771-kube-api-access-gjznb\") pod \"odh-model-controller-7998c67dc4-7kqk8\" (UID: \"8dc175df-3be1-4f44-aaed-fe6ee8e60771\") " pod="opendatahub/odh-model-controller-7998c67dc4-7kqk8" Jul 17 07:16:43.326124 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:43.326040 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-smgzc"] Jul 17 07:16:43.329543 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:43.329522 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-smgzc" Jul 17 07:16:43.331976 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:43.331950 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-mj58v\"" Jul 17 07:16:43.332113 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:43.331974 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 17 07:16:43.341152 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:43.341122 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-smgzc"] Jul 17 07:16:43.398450 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:43.398413 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42hsb\" (UniqueName: \"kubernetes.io/projected/14f64e3c-e222-4720-b723-358eeed0c402-kube-api-access-42hsb\") pod \"model-serving-api-54d89bd5fc-smgzc\" (UID: \"14f64e3c-e222-4720-b723-358eeed0c402\") " pod="opendatahub/model-serving-api-54d89bd5fc-smgzc" Jul 17 07:16:43.398632 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:43.398461 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/14f64e3c-e222-4720-b723-358eeed0c402-tls-certs\") pod \"model-serving-api-54d89bd5fc-smgzc\" (UID: \"14f64e3c-e222-4720-b723-358eeed0c402\") " pod="opendatahub/model-serving-api-54d89bd5fc-smgzc" Jul 17 07:16:43.398632 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:43.398534 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8dc175df-3be1-4f44-aaed-fe6ee8e60771-cert\") pod \"odh-model-controller-7998c67dc4-7kqk8\" (UID: \"8dc175df-3be1-4f44-aaed-fe6ee8e60771\") " pod="opendatahub/odh-model-controller-7998c67dc4-7kqk8" Jul 17 07:16:43.401001 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:43.400974 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8dc175df-3be1-4f44-aaed-fe6ee8e60771-cert\") pod \"odh-model-controller-7998c67dc4-7kqk8\" (UID: \"8dc175df-3be1-4f44-aaed-fe6ee8e60771\") " pod="opendatahub/odh-model-controller-7998c67dc4-7kqk8" Jul 17 07:16:43.498954 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:43.498918 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42hsb\" (UniqueName: \"kubernetes.io/projected/14f64e3c-e222-4720-b723-358eeed0c402-kube-api-access-42hsb\") pod \"model-serving-api-54d89bd5fc-smgzc\" (UID: \"14f64e3c-e222-4720-b723-358eeed0c402\") " pod="opendatahub/model-serving-api-54d89bd5fc-smgzc" Jul 17 07:16:43.498954 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:43.498960 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/14f64e3c-e222-4720-b723-358eeed0c402-tls-certs\") pod \"model-serving-api-54d89bd5fc-smgzc\" (UID: \"14f64e3c-e222-4720-b723-358eeed0c402\") " pod="opendatahub/model-serving-api-54d89bd5fc-smgzc" Jul 17 07:16:43.499176 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:16:43.499079 2568 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 17 07:16:43.499176 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:16:43.499131 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14f64e3c-e222-4720-b723-358eeed0c402-tls-certs podName:14f64e3c-e222-4720-b723-358eeed0c402 nodeName:}" failed. No retries permitted until 2026-07-17 07:16:43.999116977 +0000 UTC m=+308.483805930 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/14f64e3c-e222-4720-b723-358eeed0c402-tls-certs") pod "model-serving-api-54d89bd5fc-smgzc" (UID: "14f64e3c-e222-4720-b723-358eeed0c402") : secret "model-serving-api-tls" not found Jul 17 07:16:43.510314 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:43.510257 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42hsb\" (UniqueName: \"kubernetes.io/projected/14f64e3c-e222-4720-b723-358eeed0c402-kube-api-access-42hsb\") pod \"model-serving-api-54d89bd5fc-smgzc\" (UID: \"14f64e3c-e222-4720-b723-358eeed0c402\") " pod="opendatahub/model-serving-api-54d89bd5fc-smgzc" Jul 17 07:16:43.566678 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:43.566642 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-7kqk8" Jul 17 07:16:43.700085 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:43.700054 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-7kqk8"] Jul 17 07:16:43.702096 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:16:43.702069 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dc175df_3be1_4f44_aaed_fe6ee8e60771.slice/crio-f4469a3451a4f4ecd49f93644667f330c21b07510e8ad6a34e883d451e3139bc WatchSource:0}: Error finding container f4469a3451a4f4ecd49f93644667f330c21b07510e8ad6a34e883d451e3139bc: Status 404 returned error can't find the container with id f4469a3451a4f4ecd49f93644667f330c21b07510e8ad6a34e883d451e3139bc Jul 17 07:16:43.703466 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:43.703446 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 17 07:16:44.002646 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:44.002614 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/14f64e3c-e222-4720-b723-358eeed0c402-tls-certs\") pod \"model-serving-api-54d89bd5fc-smgzc\" (UID: \"14f64e3c-e222-4720-b723-358eeed0c402\") " pod="opendatahub/model-serving-api-54d89bd5fc-smgzc" Jul 17 07:16:44.005076 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:44.005041 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/14f64e3c-e222-4720-b723-358eeed0c402-tls-certs\") pod \"model-serving-api-54d89bd5fc-smgzc\" (UID: \"14f64e3c-e222-4720-b723-358eeed0c402\") " pod="opendatahub/model-serving-api-54d89bd5fc-smgzc" Jul 17 07:16:44.242689 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:44.242643 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-smgzc" Jul 17 07:16:44.377813 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:44.377781 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-smgzc"] Jul 17 07:16:44.379548 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:16:44.379512 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14f64e3c_e222_4720_b723_358eeed0c402.slice/crio-d997e5ede85a1018368e7ca9459fe95bedfde2ab540b306a394f79f30bf45a28 WatchSource:0}: Error finding container d997e5ede85a1018368e7ca9459fe95bedfde2ab540b306a394f79f30bf45a28: Status 404 returned error can't find the container with id d997e5ede85a1018368e7ca9459fe95bedfde2ab540b306a394f79f30bf45a28 Jul 17 07:16:44.401036 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:44.400998 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-7kqk8" event={"ID":"8dc175df-3be1-4f44-aaed-fe6ee8e60771","Type":"ContainerStarted","Data":"f4469a3451a4f4ecd49f93644667f330c21b07510e8ad6a34e883d451e3139bc"} Jul 17 07:16:44.402218 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:44.402197 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-smgzc" event={"ID":"14f64e3c-e222-4720-b723-358eeed0c402","Type":"ContainerStarted","Data":"d997e5ede85a1018368e7ca9459fe95bedfde2ab540b306a394f79f30bf45a28"} Jul 17 07:16:47.420359 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:47.420321 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-smgzc" event={"ID":"14f64e3c-e222-4720-b723-358eeed0c402","Type":"ContainerStarted","Data":"35c31045a8bcb048c3993344803cae27b09497a0b4aa62f3351df39eee083720"} Jul 17 07:16:47.420773 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:47.420480 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-smgzc" Jul 17 07:16:47.421855 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:47.421836 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-7kqk8" event={"ID":"8dc175df-3be1-4f44-aaed-fe6ee8e60771","Type":"ContainerStarted","Data":"ff3bf24115763619f3d3929a2726e8f796ca497a59c6cae41738daa3fd8d0923"} Jul 17 07:16:47.421921 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:47.421893 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-7kqk8" Jul 17 07:16:47.437227 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:47.437178 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-smgzc" podStartSLOduration=1.607118064 podStartE2EDuration="4.437165046s" totalImagesPullingTime="2.830046982s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 07:16:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:16:47.436358457 +0000 UTC m=+311.921047431" watchObservedRunningTime="2026-07-17 07:16:47.437165046 +0000 UTC m=+311.921854021" Jul 17 07:16:47.451792 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:47.451749 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-7kqk8" podStartSLOduration=1.941383148 podStartE2EDuration="5.451735246s" totalImagesPullingTime="3.510352098s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 07:16:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:16:47.451396009 +0000 UTC m=+311.936084983" watchObservedRunningTime="2026-07-17 07:16:47.451735246 +0000 UTC m=+311.936424221" Jul 17 07:16:48.006529 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:48.006495 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5f788c4b68-tl59q"] Jul 17 07:16:48.009826 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:48.009807 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5f788c4b68-tl59q" Jul 17 07:16:48.012200 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:48.012179 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 17 07:16:48.012389 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:48.012362 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 17 07:16:48.013110 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:48.013087 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-dbrkw\"" Jul 17 07:16:48.027450 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:48.027430 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5f788c4b68-tl59q"] Jul 17 07:16:48.140802 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:48.140771 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/89768ae7-e3bf-420a-9c1b-586009b2bff3-tmp\") pod \"kube-auth-proxy-5f788c4b68-tl59q\" (UID: \"89768ae7-e3bf-420a-9c1b-586009b2bff3\") " pod="openshift-ingress/kube-auth-proxy-5f788c4b68-tl59q" Jul 17 07:16:48.140947 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:48.140824 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89768ae7-e3bf-420a-9c1b-586009b2bff3-tls-certs\") pod \"kube-auth-proxy-5f788c4b68-tl59q\" (UID: \"89768ae7-e3bf-420a-9c1b-586009b2bff3\") " pod="openshift-ingress/kube-auth-proxy-5f788c4b68-tl59q" Jul 17 07:16:48.140947 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:48.140874 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4nh6\" (UniqueName: \"kubernetes.io/projected/89768ae7-e3bf-420a-9c1b-586009b2bff3-kube-api-access-x4nh6\") pod \"kube-auth-proxy-5f788c4b68-tl59q\" (UID: \"89768ae7-e3bf-420a-9c1b-586009b2bff3\") " pod="openshift-ingress/kube-auth-proxy-5f788c4b68-tl59q" Jul 17 07:16:48.241418 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:48.241386 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/89768ae7-e3bf-420a-9c1b-586009b2bff3-tmp\") pod \"kube-auth-proxy-5f788c4b68-tl59q\" (UID: \"89768ae7-e3bf-420a-9c1b-586009b2bff3\") " pod="openshift-ingress/kube-auth-proxy-5f788c4b68-tl59q" Jul 17 07:16:48.241604 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:48.241441 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89768ae7-e3bf-420a-9c1b-586009b2bff3-tls-certs\") pod \"kube-auth-proxy-5f788c4b68-tl59q\" (UID: \"89768ae7-e3bf-420a-9c1b-586009b2bff3\") " pod="openshift-ingress/kube-auth-proxy-5f788c4b68-tl59q" Jul 17 07:16:48.241604 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:48.241570 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4nh6\" (UniqueName: \"kubernetes.io/projected/89768ae7-e3bf-420a-9c1b-586009b2bff3-kube-api-access-x4nh6\") pod \"kube-auth-proxy-5f788c4b68-tl59q\" (UID: \"89768ae7-e3bf-420a-9c1b-586009b2bff3\") " pod="openshift-ingress/kube-auth-proxy-5f788c4b68-tl59q" Jul 17 07:16:48.243645 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:48.243615 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/89768ae7-e3bf-420a-9c1b-586009b2bff3-tmp\") pod \"kube-auth-proxy-5f788c4b68-tl59q\" (UID: \"89768ae7-e3bf-420a-9c1b-586009b2bff3\") " pod="openshift-ingress/kube-auth-proxy-5f788c4b68-tl59q" Jul 17 07:16:48.243805 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:48.243787 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89768ae7-e3bf-420a-9c1b-586009b2bff3-tls-certs\") pod \"kube-auth-proxy-5f788c4b68-tl59q\" (UID: \"89768ae7-e3bf-420a-9c1b-586009b2bff3\") " pod="openshift-ingress/kube-auth-proxy-5f788c4b68-tl59q" Jul 17 07:16:48.250439 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:48.250415 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4nh6\" (UniqueName: \"kubernetes.io/projected/89768ae7-e3bf-420a-9c1b-586009b2bff3-kube-api-access-x4nh6\") pod \"kube-auth-proxy-5f788c4b68-tl59q\" (UID: \"89768ae7-e3bf-420a-9c1b-586009b2bff3\") " pod="openshift-ingress/kube-auth-proxy-5f788c4b68-tl59q" Jul 17 07:16:48.320133 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:48.320061 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5f788c4b68-tl59q" Jul 17 07:16:48.460531 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:48.460501 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5f788c4b68-tl59q"] Jul 17 07:16:48.462614 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:16:48.462585 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89768ae7_e3bf_420a_9c1b_586009b2bff3.slice/crio-de815f2e4a627df4570b3736db2dc47fe8331f9b8c70a3e7ca82198441af9dc5 WatchSource:0}: Error finding container de815f2e4a627df4570b3736db2dc47fe8331f9b8c70a3e7ca82198441af9dc5: Status 404 returned error can't find the container with id de815f2e4a627df4570b3736db2dc47fe8331f9b8c70a3e7ca82198441af9dc5 Jul 17 07:16:49.309636 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:49.309604 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-jdb29"] Jul 17 07:16:49.313153 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:49.313132 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jdb29" Jul 17 07:16:49.315618 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:49.315580 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 17 07:16:49.315618 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:49.315606 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 17 07:16:49.315854 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:49.315603 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 17 07:16:49.316042 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:49.316024 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 17 07:16:49.316256 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:49.316233 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-lw6s6\"" Jul 17 07:16:49.330130 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:49.330107 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-jdb29"] Jul 17 07:16:49.433962 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:49.433922 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5f788c4b68-tl59q" event={"ID":"89768ae7-e3bf-420a-9c1b-586009b2bff3","Type":"ContainerStarted","Data":"de815f2e4a627df4570b3736db2dc47fe8331f9b8c70a3e7ca82198441af9dc5"} Jul 17 07:16:49.450347 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:49.450312 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0992b53b-070c-486d-9b9c-798cf3420710-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-jdb29\" (UID: \"0992b53b-070c-486d-9b9c-798cf3420710\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jdb29" Jul 17 07:16:49.450510 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:49.450367 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0992b53b-070c-486d-9b9c-798cf3420710-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-jdb29\" (UID: \"0992b53b-070c-486d-9b9c-798cf3420710\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jdb29" Jul 17 07:16:49.450510 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:49.450456 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0992b53b-070c-486d-9b9c-798cf3420710-istio-token\") pod \"istiod-openshift-gateway-7574459656-jdb29\" (UID: \"0992b53b-070c-486d-9b9c-798cf3420710\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jdb29" Jul 17 07:16:49.450510 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:49.450500 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0992b53b-070c-486d-9b9c-798cf3420710-cacerts\") pod \"istiod-openshift-gateway-7574459656-jdb29\" (UID: \"0992b53b-070c-486d-9b9c-798cf3420710\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jdb29" Jul 17 07:16:49.450671 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:49.450528 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0992b53b-070c-486d-9b9c-798cf3420710-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-jdb29\" (UID: \"0992b53b-070c-486d-9b9c-798cf3420710\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jdb29" Jul 17 07:16:49.450671 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:49.450554 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0992b53b-070c-486d-9b9c-798cf3420710-local-certs\") pod \"istiod-openshift-gateway-7574459656-jdb29\" (UID: \"0992b53b-070c-486d-9b9c-798cf3420710\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jdb29" Jul 17 07:16:49.450671 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:49.450580 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mz48g\" (UniqueName: \"kubernetes.io/projected/0992b53b-070c-486d-9b9c-798cf3420710-kube-api-access-mz48g\") pod \"istiod-openshift-gateway-7574459656-jdb29\" (UID: \"0992b53b-070c-486d-9b9c-798cf3420710\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jdb29" Jul 17 07:16:49.551940 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:49.551902 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0992b53b-070c-486d-9b9c-798cf3420710-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-jdb29\" (UID: \"0992b53b-070c-486d-9b9c-798cf3420710\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jdb29" Jul 17 07:16:49.552402 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:49.551964 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0992b53b-070c-486d-9b9c-798cf3420710-istio-token\") pod \"istiod-openshift-gateway-7574459656-jdb29\" (UID: \"0992b53b-070c-486d-9b9c-798cf3420710\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jdb29" Jul 17 07:16:49.552402 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:49.551991 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0992b53b-070c-486d-9b9c-798cf3420710-cacerts\") pod \"istiod-openshift-gateway-7574459656-jdb29\" (UID: \"0992b53b-070c-486d-9b9c-798cf3420710\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jdb29" Jul 17 07:16:49.552402 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:49.552024 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0992b53b-070c-486d-9b9c-798cf3420710-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-jdb29\" (UID: \"0992b53b-070c-486d-9b9c-798cf3420710\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jdb29" Jul 17 07:16:49.552402 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:49.552052 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0992b53b-070c-486d-9b9c-798cf3420710-local-certs\") pod \"istiod-openshift-gateway-7574459656-jdb29\" (UID: \"0992b53b-070c-486d-9b9c-798cf3420710\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jdb29" Jul 17 07:16:49.552402 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:49.552083 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mz48g\" (UniqueName: \"kubernetes.io/projected/0992b53b-070c-486d-9b9c-798cf3420710-kube-api-access-mz48g\") pod \"istiod-openshift-gateway-7574459656-jdb29\" (UID: \"0992b53b-070c-486d-9b9c-798cf3420710\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jdb29" Jul 17 07:16:49.552402 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:49.552131 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0992b53b-070c-486d-9b9c-798cf3420710-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-jdb29\" (UID: \"0992b53b-070c-486d-9b9c-798cf3420710\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jdb29" Jul 17 07:16:49.552992 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:49.552944 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0992b53b-070c-486d-9b9c-798cf3420710-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-jdb29\" (UID: \"0992b53b-070c-486d-9b9c-798cf3420710\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jdb29" Jul 17 07:16:49.554932 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:49.554903 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0992b53b-070c-486d-9b9c-798cf3420710-local-certs\") pod \"istiod-openshift-gateway-7574459656-jdb29\" (UID: \"0992b53b-070c-486d-9b9c-798cf3420710\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jdb29" Jul 17 07:16:49.555115 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:49.555089 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0992b53b-070c-486d-9b9c-798cf3420710-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-jdb29\" (UID: \"0992b53b-070c-486d-9b9c-798cf3420710\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jdb29" Jul 17 07:16:49.555236 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:49.555147 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0992b53b-070c-486d-9b9c-798cf3420710-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-jdb29\" (UID: \"0992b53b-070c-486d-9b9c-798cf3420710\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jdb29" Jul 17 07:16:49.555438 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:49.555409 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0992b53b-070c-486d-9b9c-798cf3420710-cacerts\") pod \"istiod-openshift-gateway-7574459656-jdb29\" (UID: \"0992b53b-070c-486d-9b9c-798cf3420710\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jdb29" Jul 17 07:16:49.559903 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:49.559850 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0992b53b-070c-486d-9b9c-798cf3420710-istio-token\") pod \"istiod-openshift-gateway-7574459656-jdb29\" (UID: \"0992b53b-070c-486d-9b9c-798cf3420710\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jdb29" Jul 17 07:16:49.560653 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:49.560627 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mz48g\" (UniqueName: \"kubernetes.io/projected/0992b53b-070c-486d-9b9c-798cf3420710-kube-api-access-mz48g\") pod \"istiod-openshift-gateway-7574459656-jdb29\" (UID: \"0992b53b-070c-486d-9b9c-798cf3420710\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jdb29" Jul 17 07:16:49.624053 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:49.624024 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jdb29" Jul 17 07:16:49.782174 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:49.782142 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-jdb29"] Jul 17 07:16:49.783694 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:16:49.783667 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0992b53b_070c_486d_9b9c_798cf3420710.slice/crio-8f9ec22bf9409073480702a4631548a902f052b82277bfd17e6042171cdc57c2 WatchSource:0}: Error finding container 8f9ec22bf9409073480702a4631548a902f052b82277bfd17e6042171cdc57c2: Status 404 returned error can't find the container with id 8f9ec22bf9409073480702a4631548a902f052b82277bfd17e6042171cdc57c2 Jul 17 07:16:50.440482 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:50.440448 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jdb29" event={"ID":"0992b53b-070c-486d-9b9c-798cf3420710","Type":"ContainerStarted","Data":"8f9ec22bf9409073480702a4631548a902f052b82277bfd17e6042171cdc57c2"} Jul 17 07:16:52.960751 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:52.960708 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:16:52.961044 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:52.960804 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:16:53.455943 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:53.455904 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5f788c4b68-tl59q" event={"ID":"89768ae7-e3bf-420a-9c1b-586009b2bff3","Type":"ContainerStarted","Data":"71844360902b817ac0c4715791eb51da15c9cb6e09bd6d6d7990aed59f7106f2"} Jul 17 07:16:53.457525 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:53.457494 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jdb29" event={"ID":"0992b53b-070c-486d-9b9c-798cf3420710","Type":"ContainerStarted","Data":"c49a3546665ae662c566e5f7e0816c7bf2839a2cf33e482306ab5e8a121f713d"} Jul 17 07:16:53.457663 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:53.457603 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jdb29" Jul 17 07:16:53.492294 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:53.492249 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5f788c4b68-tl59q" podStartSLOduration=1.99261102 podStartE2EDuration="6.492236288s" totalImagesPullingTime="4.499625268s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 07:16:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:16:53.490784784 +0000 UTC m=+317.975473755" watchObservedRunningTime="2026-07-17 07:16:53.492236288 +0000 UTC m=+317.976925262" Jul 17 07:16:53.714424 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:53.714314 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jdb29" podStartSLOduration=1.539543058 podStartE2EDuration="4.714295783s" totalImagesPullingTime="3.174752725s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 07:16:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:16:53.538668353 +0000 UTC m=+318.023357326" watchObservedRunningTime="2026-07-17 07:16:53.714295783 +0000 UTC m=+318.198984748" Jul 17 07:16:53.715612 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:53.715594 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-nznpn"] Jul 17 07:16:53.718870 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:53.718850 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-nznpn" Jul 17 07:16:53.721587 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:53.721566 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-wd4b9\"" Jul 17 07:16:53.721802 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:53.721786 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 17 07:16:53.731707 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:53.731684 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-kskbn"] Jul 17 07:16:53.734897 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:53.734875 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kskbn" Jul 17 07:16:53.735840 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:53.735819 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-nznpn"] Jul 17 07:16:53.737106 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:53.737089 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 17 07:16:53.737229 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:53.737212 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-txplg\"" Jul 17 07:16:53.753512 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:53.753487 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-kskbn"] Jul 17 07:16:53.800611 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:53.800580 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kz5fh\" (UniqueName: \"kubernetes.io/projected/563e3f68-d742-4e2b-8cab-03385da4780c-kube-api-access-kz5fh\") pod \"kserve-controller-manager-d6f4b65b8-nznpn\" (UID: \"563e3f68-d742-4e2b-8cab-03385da4780c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-nznpn" Jul 17 07:16:53.800723 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:53.800632 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/563e3f68-d742-4e2b-8cab-03385da4780c-cert\") pod \"kserve-controller-manager-d6f4b65b8-nznpn\" (UID: \"563e3f68-d742-4e2b-8cab-03385da4780c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-nznpn" Jul 17 07:16:53.901592 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:53.901558 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/563e3f68-d742-4e2b-8cab-03385da4780c-cert\") pod \"kserve-controller-manager-d6f4b65b8-nznpn\" (UID: \"563e3f68-d742-4e2b-8cab-03385da4780c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-nznpn" Jul 17 07:16:53.901737 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:53.901596 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mccnf\" (UniqueName: \"kubernetes.io/projected/92188035-d0d9-4c04-b496-0d53544cde37-kube-api-access-mccnf\") pod \"llmisvc-controller-manager-777c6bd7d5-kskbn\" (UID: \"92188035-d0d9-4c04-b496-0d53544cde37\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kskbn" Jul 17 07:16:53.901737 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:53.901712 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/92188035-d0d9-4c04-b496-0d53544cde37-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-kskbn\" (UID: \"92188035-d0d9-4c04-b496-0d53544cde37\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kskbn" Jul 17 07:16:53.901816 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:53.901748 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kz5fh\" (UniqueName: \"kubernetes.io/projected/563e3f68-d742-4e2b-8cab-03385da4780c-kube-api-access-kz5fh\") pod \"kserve-controller-manager-d6f4b65b8-nznpn\" (UID: \"563e3f68-d742-4e2b-8cab-03385da4780c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-nznpn" Jul 17 07:16:53.903965 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:53.903936 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/563e3f68-d742-4e2b-8cab-03385da4780c-cert\") pod \"kserve-controller-manager-d6f4b65b8-nznpn\" (UID: \"563e3f68-d742-4e2b-8cab-03385da4780c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-nznpn" Jul 17 07:16:53.946813 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:53.946781 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kz5fh\" (UniqueName: \"kubernetes.io/projected/563e3f68-d742-4e2b-8cab-03385da4780c-kube-api-access-kz5fh\") pod \"kserve-controller-manager-d6f4b65b8-nznpn\" (UID: \"563e3f68-d742-4e2b-8cab-03385da4780c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-nznpn" Jul 17 07:16:54.003170 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:54.003096 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/92188035-d0d9-4c04-b496-0d53544cde37-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-kskbn\" (UID: \"92188035-d0d9-4c04-b496-0d53544cde37\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kskbn" Jul 17 07:16:54.003584 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:54.003183 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mccnf\" (UniqueName: \"kubernetes.io/projected/92188035-d0d9-4c04-b496-0d53544cde37-kube-api-access-mccnf\") pod \"llmisvc-controller-manager-777c6bd7d5-kskbn\" (UID: \"92188035-d0d9-4c04-b496-0d53544cde37\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kskbn" Jul 17 07:16:54.005757 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:54.005734 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/92188035-d0d9-4c04-b496-0d53544cde37-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-kskbn\" (UID: \"92188035-d0d9-4c04-b496-0d53544cde37\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kskbn" Jul 17 07:16:54.018409 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:54.018390 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mccnf\" (UniqueName: \"kubernetes.io/projected/92188035-d0d9-4c04-b496-0d53544cde37-kube-api-access-mccnf\") pod \"llmisvc-controller-manager-777c6bd7d5-kskbn\" (UID: \"92188035-d0d9-4c04-b496-0d53544cde37\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kskbn" Jul 17 07:16:54.029343 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:54.029326 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-nznpn" Jul 17 07:16:54.044483 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:54.044459 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kskbn" Jul 17 07:16:54.212190 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:54.212022 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-nznpn"] Jul 17 07:16:54.212853 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:16:54.212822 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod563e3f68_d742_4e2b_8cab_03385da4780c.slice/crio-7312a754292ea94909d66c767f0325494916bfd0058e877488613bcd8c98b18c WatchSource:0}: Error finding container 7312a754292ea94909d66c767f0325494916bfd0058e877488613bcd8c98b18c: Status 404 returned error can't find the container with id 7312a754292ea94909d66c767f0325494916bfd0058e877488613bcd8c98b18c Jul 17 07:16:54.241421 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:16:54.241396 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod92188035_d0d9_4c04_b496_0d53544cde37.slice/crio-c69800633aa37c40b1341a795f088b57c40b0e3560850b2cb36ba0e56cd12f7a WatchSource:0}: Error finding container c69800633aa37c40b1341a795f088b57c40b0e3560850b2cb36ba0e56cd12f7a: Status 404 returned error can't find the container with id c69800633aa37c40b1341a795f088b57c40b0e3560850b2cb36ba0e56cd12f7a Jul 17 07:16:54.242793 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:54.242770 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-kskbn"] Jul 17 07:16:54.463439 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:54.463407 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-nznpn" event={"ID":"563e3f68-d742-4e2b-8cab-03385da4780c","Type":"ContainerStarted","Data":"7312a754292ea94909d66c767f0325494916bfd0058e877488613bcd8c98b18c"} Jul 17 07:16:54.464684 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:54.464653 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kskbn" event={"ID":"92188035-d0d9-4c04-b496-0d53544cde37","Type":"ContainerStarted","Data":"c69800633aa37c40b1341a795f088b57c40b0e3560850b2cb36ba0e56cd12f7a"} Jul 17 07:16:54.466541 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:54.466515 2568 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-jdb29 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 17 07:16:54.466630 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:54.466573 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jdb29" podUID="0992b53b-070c-486d-9b9c-798cf3420710" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 17 07:16:54.623814 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:54.623783 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnftr"] Jul 17 07:16:54.628239 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:54.628220 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnftr" Jul 17 07:16:54.630965 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:54.630945 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-s8smh\"" Jul 17 07:16:54.647194 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:54.647169 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnftr"] Jul 17 07:16:54.710734 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:54.710675 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0bef2c56-a999-46b3-96a2-8ef76f96052b-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bnftr\" (UID: \"0bef2c56-a999-46b3-96a2-8ef76f96052b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnftr" Jul 17 07:16:54.710734 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:54.710726 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmchw\" (UniqueName: \"kubernetes.io/projected/0bef2c56-a999-46b3-96a2-8ef76f96052b-kube-api-access-tmchw\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bnftr\" (UID: \"0bef2c56-a999-46b3-96a2-8ef76f96052b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnftr" Jul 17 07:16:54.710969 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:54.710803 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0bef2c56-a999-46b3-96a2-8ef76f96052b-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bnftr\" (UID: \"0bef2c56-a999-46b3-96a2-8ef76f96052b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnftr" Jul 17 07:16:54.710969 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:54.710845 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0bef2c56-a999-46b3-96a2-8ef76f96052b-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bnftr\" (UID: \"0bef2c56-a999-46b3-96a2-8ef76f96052b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnftr" Jul 17 07:16:54.710969 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:54.710886 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bef2c56-a999-46b3-96a2-8ef76f96052b-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bnftr\" (UID: \"0bef2c56-a999-46b3-96a2-8ef76f96052b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnftr" Jul 17 07:16:54.711125 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:54.711007 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0bef2c56-a999-46b3-96a2-8ef76f96052b-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bnftr\" (UID: \"0bef2c56-a999-46b3-96a2-8ef76f96052b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnftr" Jul 17 07:16:54.711125 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:54.711061 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bef2c56-a999-46b3-96a2-8ef76f96052b-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bnftr\" (UID: \"0bef2c56-a999-46b3-96a2-8ef76f96052b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnftr" Jul 17 07:16:54.711125 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:54.711086 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0bef2c56-a999-46b3-96a2-8ef76f96052b-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bnftr\" (UID: \"0bef2c56-a999-46b3-96a2-8ef76f96052b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnftr" Jul 17 07:16:54.711311 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:54.711134 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0bef2c56-a999-46b3-96a2-8ef76f96052b-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bnftr\" (UID: \"0bef2c56-a999-46b3-96a2-8ef76f96052b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnftr" Jul 17 07:16:54.713119 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:54.713097 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-jdb29"] Jul 17 07:16:54.812679 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:54.812602 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0bef2c56-a999-46b3-96a2-8ef76f96052b-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bnftr\" (UID: \"0bef2c56-a999-46b3-96a2-8ef76f96052b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnftr" Jul 17 07:16:54.812679 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:54.812666 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bef2c56-a999-46b3-96a2-8ef76f96052b-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bnftr\" (UID: \"0bef2c56-a999-46b3-96a2-8ef76f96052b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnftr" Jul 17 07:16:54.812889 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:54.812720 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0bef2c56-a999-46b3-96a2-8ef76f96052b-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bnftr\" (UID: \"0bef2c56-a999-46b3-96a2-8ef76f96052b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnftr" Jul 17 07:16:54.812889 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:54.812764 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bef2c56-a999-46b3-96a2-8ef76f96052b-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bnftr\" (UID: \"0bef2c56-a999-46b3-96a2-8ef76f96052b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnftr" Jul 17 07:16:54.812889 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:54.812787 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0bef2c56-a999-46b3-96a2-8ef76f96052b-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bnftr\" (UID: \"0bef2c56-a999-46b3-96a2-8ef76f96052b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnftr" Jul 17 07:16:54.812889 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:54.812842 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0bef2c56-a999-46b3-96a2-8ef76f96052b-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bnftr\" (UID: \"0bef2c56-a999-46b3-96a2-8ef76f96052b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnftr" Jul 17 07:16:54.812889 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:54.812880 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0bef2c56-a999-46b3-96a2-8ef76f96052b-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bnftr\" (UID: \"0bef2c56-a999-46b3-96a2-8ef76f96052b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnftr" Jul 17 07:16:54.813158 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:54.812904 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmchw\" (UniqueName: \"kubernetes.io/projected/0bef2c56-a999-46b3-96a2-8ef76f96052b-kube-api-access-tmchw\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bnftr\" (UID: \"0bef2c56-a999-46b3-96a2-8ef76f96052b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnftr" Jul 17 07:16:54.813158 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:54.812973 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0bef2c56-a999-46b3-96a2-8ef76f96052b-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bnftr\" (UID: \"0bef2c56-a999-46b3-96a2-8ef76f96052b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnftr" Jul 17 07:16:54.813266 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:54.813151 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bef2c56-a999-46b3-96a2-8ef76f96052b-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bnftr\" (UID: \"0bef2c56-a999-46b3-96a2-8ef76f96052b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnftr" Jul 17 07:16:54.813559 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:54.813389 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bef2c56-a999-46b3-96a2-8ef76f96052b-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bnftr\" (UID: \"0bef2c56-a999-46b3-96a2-8ef76f96052b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnftr" Jul 17 07:16:54.813559 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:54.813464 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0bef2c56-a999-46b3-96a2-8ef76f96052b-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bnftr\" (UID: \"0bef2c56-a999-46b3-96a2-8ef76f96052b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnftr" Jul 17 07:16:54.813559 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:54.813538 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0bef2c56-a999-46b3-96a2-8ef76f96052b-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bnftr\" (UID: \"0bef2c56-a999-46b3-96a2-8ef76f96052b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnftr" Jul 17 07:16:54.813850 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:54.813824 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0bef2c56-a999-46b3-96a2-8ef76f96052b-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bnftr\" (UID: \"0bef2c56-a999-46b3-96a2-8ef76f96052b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnftr" Jul 17 07:16:54.815433 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:54.815401 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0bef2c56-a999-46b3-96a2-8ef76f96052b-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bnftr\" (UID: \"0bef2c56-a999-46b3-96a2-8ef76f96052b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnftr" Jul 17 07:16:54.816299 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:54.816262 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0bef2c56-a999-46b3-96a2-8ef76f96052b-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bnftr\" (UID: \"0bef2c56-a999-46b3-96a2-8ef76f96052b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnftr" Jul 17 07:16:54.835454 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:54.835432 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0bef2c56-a999-46b3-96a2-8ef76f96052b-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bnftr\" (UID: \"0bef2c56-a999-46b3-96a2-8ef76f96052b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnftr" Jul 17 07:16:54.839655 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:54.839632 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmchw\" (UniqueName: \"kubernetes.io/projected/0bef2c56-a999-46b3-96a2-8ef76f96052b-kube-api-access-tmchw\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bnftr\" (UID: \"0bef2c56-a999-46b3-96a2-8ef76f96052b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnftr" Jul 17 07:16:54.939455 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:54.939420 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnftr" Jul 17 07:16:55.098096 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:55.098043 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnftr"] Jul 17 07:16:55.099557 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:16:55.099520 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bef2c56_a999_46b3_96a2_8ef76f96052b.slice/crio-63c143684ab198ebf4795d961679799733877efaa7efc41a1e64e39348ceb0e2 WatchSource:0}: Error finding container 63c143684ab198ebf4795d961679799733877efaa7efc41a1e64e39348ceb0e2: Status 404 returned error can't find the container with id 63c143684ab198ebf4795d961679799733877efaa7efc41a1e64e39348ceb0e2 Jul 17 07:16:55.472805 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:55.472773 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnftr" event={"ID":"0bef2c56-a999-46b3-96a2-8ef76f96052b","Type":"ContainerStarted","Data":"63c143684ab198ebf4795d961679799733877efaa7efc41a1e64e39348ceb0e2"} Jul 17 07:16:55.473415 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:55.473382 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jdb29" podUID="0992b53b-070c-486d-9b9c-798cf3420710" containerName="discovery" containerID="cri-o://c49a3546665ae662c566e5f7e0816c7bf2839a2cf33e482306ab5e8a121f713d" gracePeriod=30 Jul 17 07:16:55.473905 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:55.473864 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jdb29" Jul 17 07:16:55.762941 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:55.762909 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jdb29" Jul 17 07:16:55.925982 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:55.924911 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0992b53b-070c-486d-9b9c-798cf3420710-cacerts\") pod \"0992b53b-070c-486d-9b9c-798cf3420710\" (UID: \"0992b53b-070c-486d-9b9c-798cf3420710\") " Jul 17 07:16:55.925982 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:55.924981 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0992b53b-070c-486d-9b9c-798cf3420710-istio-token\") pod \"0992b53b-070c-486d-9b9c-798cf3420710\" (UID: \"0992b53b-070c-486d-9b9c-798cf3420710\") " Jul 17 07:16:55.925982 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:55.925019 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0992b53b-070c-486d-9b9c-798cf3420710-istio-csr-dns-cert\") pod \"0992b53b-070c-486d-9b9c-798cf3420710\" (UID: \"0992b53b-070c-486d-9b9c-798cf3420710\") " Jul 17 07:16:55.925982 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:55.925044 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mz48g\" (UniqueName: \"kubernetes.io/projected/0992b53b-070c-486d-9b9c-798cf3420710-kube-api-access-mz48g\") pod \"0992b53b-070c-486d-9b9c-798cf3420710\" (UID: \"0992b53b-070c-486d-9b9c-798cf3420710\") " Jul 17 07:16:55.925982 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:55.925075 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0992b53b-070c-486d-9b9c-798cf3420710-local-certs\") pod \"0992b53b-070c-486d-9b9c-798cf3420710\" (UID: \"0992b53b-070c-486d-9b9c-798cf3420710\") " Jul 17 07:16:55.925982 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:55.925101 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0992b53b-070c-486d-9b9c-798cf3420710-istio-kubeconfig\") pod \"0992b53b-070c-486d-9b9c-798cf3420710\" (UID: \"0992b53b-070c-486d-9b9c-798cf3420710\") " Jul 17 07:16:55.925982 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:55.925132 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0992b53b-070c-486d-9b9c-798cf3420710-istio-csr-ca-configmap\") pod \"0992b53b-070c-486d-9b9c-798cf3420710\" (UID: \"0992b53b-070c-486d-9b9c-798cf3420710\") " Jul 17 07:16:55.926897 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:55.926843 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0992b53b-070c-486d-9b9c-798cf3420710-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "0992b53b-070c-486d-9b9c-798cf3420710" (UID: "0992b53b-070c-486d-9b9c-798cf3420710"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 07:16:55.928608 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:55.928582 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0992b53b-070c-486d-9b9c-798cf3420710-kube-api-access-mz48g" (OuterVolumeSpecName: "kube-api-access-mz48g") pod "0992b53b-070c-486d-9b9c-798cf3420710" (UID: "0992b53b-070c-486d-9b9c-798cf3420710"). InnerVolumeSpecName "kube-api-access-mz48g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 07:16:55.928828 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:55.928791 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0992b53b-070c-486d-9b9c-798cf3420710-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "0992b53b-070c-486d-9b9c-798cf3420710" (UID: "0992b53b-070c-486d-9b9c-798cf3420710"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 07:16:55.929058 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:55.929036 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0992b53b-070c-486d-9b9c-798cf3420710-cacerts" (OuterVolumeSpecName: "cacerts") pod "0992b53b-070c-486d-9b9c-798cf3420710" (UID: "0992b53b-070c-486d-9b9c-798cf3420710"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 07:16:55.929637 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:55.929599 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0992b53b-070c-486d-9b9c-798cf3420710-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "0992b53b-070c-486d-9b9c-798cf3420710" (UID: "0992b53b-070c-486d-9b9c-798cf3420710"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 07:16:55.929738 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:55.929672 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0992b53b-070c-486d-9b9c-798cf3420710-istio-token" (OuterVolumeSpecName: "istio-token") pod "0992b53b-070c-486d-9b9c-798cf3420710" (UID: "0992b53b-070c-486d-9b9c-798cf3420710"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 07:16:55.930325 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:55.930278 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0992b53b-070c-486d-9b9c-798cf3420710-local-certs" (OuterVolumeSpecName: "local-certs") pod "0992b53b-070c-486d-9b9c-798cf3420710" (UID: "0992b53b-070c-486d-9b9c-798cf3420710"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:16:56.026378 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:56.026351 2568 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0992b53b-070c-486d-9b9c-798cf3420710-istio-csr-dns-cert\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:16:56.026378 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:56.026379 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mz48g\" (UniqueName: \"kubernetes.io/projected/0992b53b-070c-486d-9b9c-798cf3420710-kube-api-access-mz48g\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:16:56.026542 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:56.026393 2568 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0992b53b-070c-486d-9b9c-798cf3420710-local-certs\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:16:56.026542 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:56.026406 2568 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0992b53b-070c-486d-9b9c-798cf3420710-istio-kubeconfig\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:16:56.026542 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:56.026417 2568 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0992b53b-070c-486d-9b9c-798cf3420710-istio-csr-ca-configmap\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:16:56.026542 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:56.026429 2568 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0992b53b-070c-486d-9b9c-798cf3420710-cacerts\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:16:56.026542 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:56.026441 2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0992b53b-070c-486d-9b9c-798cf3420710-istio-token\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:16:56.480592 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:56.480554 2568 generic.go:356] "Generic (PLEG): container finished" podID="0992b53b-070c-486d-9b9c-798cf3420710" containerID="c49a3546665ae662c566e5f7e0816c7bf2839a2cf33e482306ab5e8a121f713d" exitCode=0 Jul 17 07:16:56.481031 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:56.480601 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jdb29" event={"ID":"0992b53b-070c-486d-9b9c-798cf3420710","Type":"ContainerDied","Data":"c49a3546665ae662c566e5f7e0816c7bf2839a2cf33e482306ab5e8a121f713d"} Jul 17 07:16:56.481031 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:56.480704 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jdb29" Jul 17 07:16:56.481031 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:56.480713 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jdb29" event={"ID":"0992b53b-070c-486d-9b9c-798cf3420710","Type":"ContainerDied","Data":"8f9ec22bf9409073480702a4631548a902f052b82277bfd17e6042171cdc57c2"} Jul 17 07:16:56.481031 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:56.480743 2568 scope.go:117] "RemoveContainer" containerID="c49a3546665ae662c566e5f7e0816c7bf2839a2cf33e482306ab5e8a121f713d" Jul 17 07:16:56.504751 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:56.504716 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-jdb29"] Jul 17 07:16:56.509762 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:56.509736 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-jdb29"] Jul 17 07:16:57.080203 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:57.080176 2568 scope.go:117] "RemoveContainer" containerID="c49a3546665ae662c566e5f7e0816c7bf2839a2cf33e482306ab5e8a121f713d" Jul 17 07:16:57.080569 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:16:57.080545 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c49a3546665ae662c566e5f7e0816c7bf2839a2cf33e482306ab5e8a121f713d\": container with ID starting with c49a3546665ae662c566e5f7e0816c7bf2839a2cf33e482306ab5e8a121f713d not found: ID does not exist" containerID="c49a3546665ae662c566e5f7e0816c7bf2839a2cf33e482306ab5e8a121f713d" Jul 17 07:16:57.080626 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:57.080582 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c49a3546665ae662c566e5f7e0816c7bf2839a2cf33e482306ab5e8a121f713d"} err="failed to get container status \"c49a3546665ae662c566e5f7e0816c7bf2839a2cf33e482306ab5e8a121f713d\": rpc error: code = NotFound desc = could not find container \"c49a3546665ae662c566e5f7e0816c7bf2839a2cf33e482306ab5e8a121f713d\": container with ID starting with c49a3546665ae662c566e5f7e0816c7bf2839a2cf33e482306ab5e8a121f713d not found: ID does not exist" Jul 17 07:16:58.000845 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:58.000809 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0992b53b-070c-486d-9b9c-798cf3420710" path="/var/lib/kubelet/pods/0992b53b-070c-486d-9b9c-798cf3420710/volumes" Jul 17 07:16:58.429617 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:58.429589 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-7kqk8" Jul 17 07:16:58.431766 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:58.431743 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-smgzc" Jul 17 07:16:58.845865 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:58.845832 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:16:58.845948 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:58.845905 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:16:58.845948 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:58.845934 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:16:59.499387 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:59.499350 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-nznpn" event={"ID":"563e3f68-d742-4e2b-8cab-03385da4780c","Type":"ContainerStarted","Data":"508e433528c5d273dbc1c55af447d007eab527990180972fdaf7c182738b51ea"} Jul 17 07:16:59.499844 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:59.499462 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-nznpn" Jul 17 07:16:59.501077 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:59.501049 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnftr" event={"ID":"0bef2c56-a999-46b3-96a2-8ef76f96052b","Type":"ContainerStarted","Data":"f96705bbf4108627858caf486cab2ddca8558d786f7b0b8bb825280bdff2a30b"} Jul 17 07:16:59.502628 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:59.502605 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kskbn" event={"ID":"92188035-d0d9-4c04-b496-0d53544cde37","Type":"ContainerStarted","Data":"15e22fdd4f1f7d8d0661a9e93f3824549ce25b8b50fc2cf79ebd431598509cc0"} Jul 17 07:16:59.502744 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:59.502656 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kskbn" Jul 17 07:16:59.531684 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:59.531633 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-nznpn" podStartSLOduration=1.9045446259999999 podStartE2EDuration="6.531619291s" totalImagesPullingTime="4.627074665s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 07:16:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:16:59.53109578 +0000 UTC m=+324.015784745" watchObservedRunningTime="2026-07-17 07:16:59.531619291 +0000 UTC m=+324.016308265" Jul 17 07:16:59.581898 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:59.581843 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnftr" podStartSLOduration=1.838261854 podStartE2EDuration="5.581806634s" totalImagesPullingTime="3.74354478s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 07:16:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:16:59.580771679 +0000 UTC m=+324.065460654" watchObservedRunningTime="2026-07-17 07:16:59.581806634 +0000 UTC m=+324.066495607" Jul 17 07:16:59.625232 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:59.625161 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kskbn" podStartSLOduration=2.019088552 podStartE2EDuration="6.625142566s" totalImagesPullingTime="4.606054014s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 07:16:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:16:59.624397869 +0000 UTC m=+324.109086844" watchObservedRunningTime="2026-07-17 07:16:59.625142566 +0000 UTC m=+324.109831540" Jul 17 07:16:59.939756 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:59.939718 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnftr" Jul 17 07:16:59.944330 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:16:59.944310 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnftr" Jul 17 07:17:00.507280 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:00.507247 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnftr" Jul 17 07:17:00.508187 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:00.508165 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnftr" Jul 17 07:17:30.509553 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:30.509461 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kskbn" Jul 17 07:17:30.512554 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:30.512528 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-nznpn" Jul 17 07:17:43.389617 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:43.389580 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-snslc"] Jul 17 07:17:43.390021 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:43.389927 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0992b53b-070c-486d-9b9c-798cf3420710" containerName="discovery" Jul 17 07:17:43.390021 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:43.389939 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="0992b53b-070c-486d-9b9c-798cf3420710" containerName="discovery" Jul 17 07:17:43.390021 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:43.389987 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="0992b53b-070c-486d-9b9c-798cf3420710" containerName="discovery" Jul 17 07:17:43.396504 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:43.396479 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-snslc" Jul 17 07:17:43.398895 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:43.398866 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 17 07:17:43.399783 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:43.399750 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 17 07:17:43.399783 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:43.399772 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-kbw87\"" Jul 17 07:17:43.403777 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:43.403750 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-snslc"] Jul 17 07:17:43.496809 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:43.496771 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtx5f\" (UniqueName: \"kubernetes.io/projected/6f973cfa-ab90-46c7-b48f-0f763d6d1338-kube-api-access-xtx5f\") pod \"kuadrant-operator-catalog-snslc\" (UID: \"6f973cfa-ab90-46c7-b48f-0f763d6d1338\") " pod="kuadrant-system/kuadrant-operator-catalog-snslc" Jul 17 07:17:43.598077 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:43.598042 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtx5f\" (UniqueName: \"kubernetes.io/projected/6f973cfa-ab90-46c7-b48f-0f763d6d1338-kube-api-access-xtx5f\") pod \"kuadrant-operator-catalog-snslc\" (UID: \"6f973cfa-ab90-46c7-b48f-0f763d6d1338\") " pod="kuadrant-system/kuadrant-operator-catalog-snslc" Jul 17 07:17:43.606535 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:43.606507 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtx5f\" (UniqueName: \"kubernetes.io/projected/6f973cfa-ab90-46c7-b48f-0f763d6d1338-kube-api-access-xtx5f\") pod \"kuadrant-operator-catalog-snslc\" (UID: \"6f973cfa-ab90-46c7-b48f-0f763d6d1338\") " pod="kuadrant-system/kuadrant-operator-catalog-snslc" Jul 17 07:17:43.708357 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:43.708258 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-snslc" Jul 17 07:17:43.755663 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:43.755631 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-snslc"] Jul 17 07:17:43.858350 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:43.858325 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-snslc"] Jul 17 07:17:43.860589 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:17:43.860562 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f973cfa_ab90_46c7_b48f_0f763d6d1338.slice/crio-cbe7fd6a7a09221dbcc2be0882f415c3a0a0e08a63142b40e6b9cd0cc2766e2b WatchSource:0}: Error finding container cbe7fd6a7a09221dbcc2be0882f415c3a0a0e08a63142b40e6b9cd0cc2766e2b: Status 404 returned error can't find the container with id cbe7fd6a7a09221dbcc2be0882f415c3a0a0e08a63142b40e6b9cd0cc2766e2b Jul 17 07:17:43.961573 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:43.961505 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-n5x8d"] Jul 17 07:17:43.966059 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:43.966040 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-n5x8d" Jul 17 07:17:43.972278 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:43.972258 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-n5x8d"] Jul 17 07:17:44.104782 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:44.104755 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwnmb\" (UniqueName: \"kubernetes.io/projected/0f3efd89-40d2-41bb-b3ba-6dafc021373b-kube-api-access-dwnmb\") pod \"kuadrant-operator-catalog-n5x8d\" (UID: \"0f3efd89-40d2-41bb-b3ba-6dafc021373b\") " pod="kuadrant-system/kuadrant-operator-catalog-n5x8d" Jul 17 07:17:44.205642 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:44.205614 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwnmb\" (UniqueName: \"kubernetes.io/projected/0f3efd89-40d2-41bb-b3ba-6dafc021373b-kube-api-access-dwnmb\") pod \"kuadrant-operator-catalog-n5x8d\" (UID: \"0f3efd89-40d2-41bb-b3ba-6dafc021373b\") " pod="kuadrant-system/kuadrant-operator-catalog-n5x8d" Jul 17 07:17:44.213379 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:44.213320 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwnmb\" (UniqueName: \"kubernetes.io/projected/0f3efd89-40d2-41bb-b3ba-6dafc021373b-kube-api-access-dwnmb\") pod \"kuadrant-operator-catalog-n5x8d\" (UID: \"0f3efd89-40d2-41bb-b3ba-6dafc021373b\") " pod="kuadrant-system/kuadrant-operator-catalog-n5x8d" Jul 17 07:17:44.276544 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:44.276514 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-n5x8d" Jul 17 07:17:44.410001 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:44.409972 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-n5x8d"] Jul 17 07:17:44.412324 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:17:44.412297 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f3efd89_40d2_41bb_b3ba_6dafc021373b.slice/crio-7970743d5e13bbfe3773e0e5c4e82b8b61c7a51d5f2e866b4dae59cb6e81b7bb WatchSource:0}: Error finding container 7970743d5e13bbfe3773e0e5c4e82b8b61c7a51d5f2e866b4dae59cb6e81b7bb: Status 404 returned error can't find the container with id 7970743d5e13bbfe3773e0e5c4e82b8b61c7a51d5f2e866b4dae59cb6e81b7bb Jul 17 07:17:44.707444 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:44.707412 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-snslc" event={"ID":"6f973cfa-ab90-46c7-b48f-0f763d6d1338","Type":"ContainerStarted","Data":"cbe7fd6a7a09221dbcc2be0882f415c3a0a0e08a63142b40e6b9cd0cc2766e2b"} Jul 17 07:17:44.708676 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:44.708649 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-n5x8d" event={"ID":"0f3efd89-40d2-41bb-b3ba-6dafc021373b","Type":"ContainerStarted","Data":"7970743d5e13bbfe3773e0e5c4e82b8b61c7a51d5f2e866b4dae59cb6e81b7bb"} Jul 17 07:17:46.722369 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:46.722258 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-snslc" event={"ID":"6f973cfa-ab90-46c7-b48f-0f763d6d1338","Type":"ContainerStarted","Data":"c27265636df163b1b069f834370ebdcce43d15ac11ac918a9361f9bc0ca193b9"} Jul 17 07:17:46.722369 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:46.722337 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-snslc" podUID="6f973cfa-ab90-46c7-b48f-0f763d6d1338" containerName="registry-server" containerID="cri-o://c27265636df163b1b069f834370ebdcce43d15ac11ac918a9361f9bc0ca193b9" gracePeriod=2 Jul 17 07:17:46.723974 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:46.723946 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-n5x8d" event={"ID":"0f3efd89-40d2-41bb-b3ba-6dafc021373b","Type":"ContainerStarted","Data":"b6e345c59c690b59fcc2a79abdb542e9150f828996cdc52aa1f2144dfd915d1c"} Jul 17 07:17:46.739747 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:46.739707 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-snslc" podStartSLOduration=1.455577606 podStartE2EDuration="3.739693604s" totalImagesPullingTime="2.284115998s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 07:17:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:17:46.739169921 +0000 UTC m=+371.223858898" watchObservedRunningTime="2026-07-17 07:17:46.739693604 +0000 UTC m=+371.224382577" Jul 17 07:17:46.761171 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:46.761124 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-n5x8d" podStartSLOduration=1.7766541839999999 podStartE2EDuration="3.761109277s" totalImagesPullingTime="1.984455093s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 07:17:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:17:46.760067704 +0000 UTC m=+371.244756678" watchObservedRunningTime="2026-07-17 07:17:46.761109277 +0000 UTC m=+371.245798250" Jul 17 07:17:46.973622 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:46.973569 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-snslc" Jul 17 07:17:47.031754 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:47.031724 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xtx5f\" (UniqueName: \"kubernetes.io/projected/6f973cfa-ab90-46c7-b48f-0f763d6d1338-kube-api-access-xtx5f\") pod \"6f973cfa-ab90-46c7-b48f-0f763d6d1338\" (UID: \"6f973cfa-ab90-46c7-b48f-0f763d6d1338\") " Jul 17 07:17:47.033870 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:47.033849 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f973cfa-ab90-46c7-b48f-0f763d6d1338-kube-api-access-xtx5f" (OuterVolumeSpecName: "kube-api-access-xtx5f") pod "6f973cfa-ab90-46c7-b48f-0f763d6d1338" (UID: "6f973cfa-ab90-46c7-b48f-0f763d6d1338"). InnerVolumeSpecName "kube-api-access-xtx5f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 07:17:47.133043 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:47.133017 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xtx5f\" (UniqueName: \"kubernetes.io/projected/6f973cfa-ab90-46c7-b48f-0f763d6d1338-kube-api-access-xtx5f\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:17:47.729447 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:47.729408 2568 generic.go:356] "Generic (PLEG): container finished" podID="6f973cfa-ab90-46c7-b48f-0f763d6d1338" containerID="c27265636df163b1b069f834370ebdcce43d15ac11ac918a9361f9bc0ca193b9" exitCode=0 Jul 17 07:17:47.729905 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:47.729461 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-snslc" Jul 17 07:17:47.729905 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:47.729494 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-snslc" event={"ID":"6f973cfa-ab90-46c7-b48f-0f763d6d1338","Type":"ContainerDied","Data":"c27265636df163b1b069f834370ebdcce43d15ac11ac918a9361f9bc0ca193b9"} Jul 17 07:17:47.729905 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:47.729548 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-snslc" event={"ID":"6f973cfa-ab90-46c7-b48f-0f763d6d1338","Type":"ContainerDied","Data":"cbe7fd6a7a09221dbcc2be0882f415c3a0a0e08a63142b40e6b9cd0cc2766e2b"} Jul 17 07:17:47.729905 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:47.729571 2568 scope.go:117] "RemoveContainer" containerID="c27265636df163b1b069f834370ebdcce43d15ac11ac918a9361f9bc0ca193b9" Jul 17 07:17:47.739499 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:47.739480 2568 scope.go:117] "RemoveContainer" containerID="c27265636df163b1b069f834370ebdcce43d15ac11ac918a9361f9bc0ca193b9" Jul 17 07:17:47.739728 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:17:47.739708 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c27265636df163b1b069f834370ebdcce43d15ac11ac918a9361f9bc0ca193b9\": container with ID starting with c27265636df163b1b069f834370ebdcce43d15ac11ac918a9361f9bc0ca193b9 not found: ID does not exist" containerID="c27265636df163b1b069f834370ebdcce43d15ac11ac918a9361f9bc0ca193b9" Jul 17 07:17:47.739772 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:47.739736 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c27265636df163b1b069f834370ebdcce43d15ac11ac918a9361f9bc0ca193b9"} err="failed to get container status \"c27265636df163b1b069f834370ebdcce43d15ac11ac918a9361f9bc0ca193b9\": rpc error: code = NotFound desc = could not find container \"c27265636df163b1b069f834370ebdcce43d15ac11ac918a9361f9bc0ca193b9\": container with ID starting with c27265636df163b1b069f834370ebdcce43d15ac11ac918a9361f9bc0ca193b9 not found: ID does not exist" Jul 17 07:17:47.752687 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:47.752660 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-snslc"] Jul 17 07:17:47.759775 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:47.759754 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-snslc"] Jul 17 07:17:47.999087 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:47.999007 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f973cfa-ab90-46c7-b48f-0f763d6d1338" path="/var/lib/kubelet/pods/6f973cfa-ab90-46c7-b48f-0f763d6d1338/volumes" Jul 17 07:17:54.277145 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:54.277103 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-n5x8d" Jul 17 07:17:54.277592 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:54.277170 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-n5x8d" Jul 17 07:17:54.299661 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:54.299638 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-n5x8d" Jul 17 07:17:54.783276 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:54.783249 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-n5x8d" Jul 17 07:17:59.603391 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:59.603353 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066lbr"] Jul 17 07:17:59.603797 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:59.603701 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f973cfa-ab90-46c7-b48f-0f763d6d1338" containerName="registry-server" Jul 17 07:17:59.603797 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:59.603712 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f973cfa-ab90-46c7-b48f-0f763d6d1338" containerName="registry-server" Jul 17 07:17:59.603797 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:59.603786 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f973cfa-ab90-46c7-b48f-0f763d6d1338" containerName="registry-server" Jul 17 07:17:59.607020 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:59.607004 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066lbr" Jul 17 07:17:59.609584 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:59.609560 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-vqgqw\"" Jul 17 07:17:59.616169 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:59.616146 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066lbr"] Jul 17 07:17:59.736647 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:59.736613 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1448f170-548f-4e5d-a909-d1c97039252a-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066lbr\" (UID: \"1448f170-548f-4e5d-a909-d1c97039252a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066lbr" Jul 17 07:17:59.736817 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:59.736651 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vn4db\" (UniqueName: \"kubernetes.io/projected/1448f170-548f-4e5d-a909-d1c97039252a-kube-api-access-vn4db\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066lbr\" (UID: \"1448f170-548f-4e5d-a909-d1c97039252a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066lbr" Jul 17 07:17:59.736817 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:59.736690 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1448f170-548f-4e5d-a909-d1c97039252a-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066lbr\" (UID: \"1448f170-548f-4e5d-a909-d1c97039252a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066lbr" Jul 17 07:17:59.837650 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:59.837610 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1448f170-548f-4e5d-a909-d1c97039252a-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066lbr\" (UID: \"1448f170-548f-4e5d-a909-d1c97039252a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066lbr" Jul 17 07:17:59.837650 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:59.837650 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vn4db\" (UniqueName: \"kubernetes.io/projected/1448f170-548f-4e5d-a909-d1c97039252a-kube-api-access-vn4db\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066lbr\" (UID: \"1448f170-548f-4e5d-a909-d1c97039252a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066lbr" Jul 17 07:17:59.837881 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:59.837700 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1448f170-548f-4e5d-a909-d1c97039252a-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066lbr\" (UID: \"1448f170-548f-4e5d-a909-d1c97039252a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066lbr" Jul 17 07:17:59.838073 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:59.838054 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1448f170-548f-4e5d-a909-d1c97039252a-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066lbr\" (UID: \"1448f170-548f-4e5d-a909-d1c97039252a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066lbr" Jul 17 07:17:59.838073 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:59.838065 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1448f170-548f-4e5d-a909-d1c97039252a-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066lbr\" (UID: \"1448f170-548f-4e5d-a909-d1c97039252a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066lbr" Jul 17 07:17:59.847910 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:59.847886 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vn4db\" (UniqueName: \"kubernetes.io/projected/1448f170-548f-4e5d-a909-d1c97039252a-kube-api-access-vn4db\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066lbr\" (UID: \"1448f170-548f-4e5d-a909-d1c97039252a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066lbr" Jul 17 07:17:59.916726 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:17:59.916698 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066lbr" Jul 17 07:18:00.057767 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:00.057733 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066lbr"] Jul 17 07:18:00.061877 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:18:00.061844 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1448f170_548f_4e5d_a909_d1c97039252a.slice/crio-ca25d89b5910598bc5e41025ec667d3ddfbf0de2d492de4e4985434bbbd0ba0b WatchSource:0}: Error finding container ca25d89b5910598bc5e41025ec667d3ddfbf0de2d492de4e4985434bbbd0ba0b: Status 404 returned error can't find the container with id ca25d89b5910598bc5e41025ec667d3ddfbf0de2d492de4e4985434bbbd0ba0b Jul 17 07:18:00.210182 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:00.210118 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599rfrw"] Jul 17 07:18:00.213572 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:00.213552 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599rfrw" Jul 17 07:18:00.223360 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:00.223334 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599rfrw"] Jul 17 07:18:00.342006 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:00.341970 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eb41a9f6-2e49-48dc-aee9-737d3b981c3e-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599rfrw\" (UID: \"eb41a9f6-2e49-48dc-aee9-737d3b981c3e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599rfrw" Jul 17 07:18:00.342006 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:00.342011 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7rnd\" (UniqueName: \"kubernetes.io/projected/eb41a9f6-2e49-48dc-aee9-737d3b981c3e-kube-api-access-k7rnd\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599rfrw\" (UID: \"eb41a9f6-2e49-48dc-aee9-737d3b981c3e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599rfrw" Jul 17 07:18:00.342214 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:00.342116 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eb41a9f6-2e49-48dc-aee9-737d3b981c3e-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599rfrw\" (UID: \"eb41a9f6-2e49-48dc-aee9-737d3b981c3e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599rfrw" Jul 17 07:18:00.443059 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:00.443021 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eb41a9f6-2e49-48dc-aee9-737d3b981c3e-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599rfrw\" (UID: \"eb41a9f6-2e49-48dc-aee9-737d3b981c3e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599rfrw" Jul 17 07:18:00.443239 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:00.443065 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7rnd\" (UniqueName: \"kubernetes.io/projected/eb41a9f6-2e49-48dc-aee9-737d3b981c3e-kube-api-access-k7rnd\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599rfrw\" (UID: \"eb41a9f6-2e49-48dc-aee9-737d3b981c3e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599rfrw" Jul 17 07:18:00.443239 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:00.443151 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eb41a9f6-2e49-48dc-aee9-737d3b981c3e-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599rfrw\" (UID: \"eb41a9f6-2e49-48dc-aee9-737d3b981c3e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599rfrw" Jul 17 07:18:00.443512 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:00.443490 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eb41a9f6-2e49-48dc-aee9-737d3b981c3e-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599rfrw\" (UID: \"eb41a9f6-2e49-48dc-aee9-737d3b981c3e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599rfrw" Jul 17 07:18:00.443584 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:00.443535 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eb41a9f6-2e49-48dc-aee9-737d3b981c3e-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599rfrw\" (UID: \"eb41a9f6-2e49-48dc-aee9-737d3b981c3e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599rfrw" Jul 17 07:18:00.452305 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:00.452273 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7rnd\" (UniqueName: \"kubernetes.io/projected/eb41a9f6-2e49-48dc-aee9-737d3b981c3e-kube-api-access-k7rnd\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599rfrw\" (UID: \"eb41a9f6-2e49-48dc-aee9-737d3b981c3e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599rfrw" Jul 17 07:18:00.536433 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:00.536368 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599rfrw" Jul 17 07:18:00.617918 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:00.617888 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t49tn"] Jul 17 07:18:00.625705 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:00.625683 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t49tn" Jul 17 07:18:00.637188 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:00.637156 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t49tn"] Jul 17 07:18:00.670399 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:00.670297 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599rfrw"] Jul 17 07:18:00.672911 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:18:00.672888 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb41a9f6_2e49_48dc_aee9_737d3b981c3e.slice/crio-4dbd0501d76fd47a34aeddd5498bfcce67aa499dfb215d35dd9ef8c20b944186 WatchSource:0}: Error finding container 4dbd0501d76fd47a34aeddd5498bfcce67aa499dfb215d35dd9ef8c20b944186: Status 404 returned error can't find the container with id 4dbd0501d76fd47a34aeddd5498bfcce67aa499dfb215d35dd9ef8c20b944186 Jul 17 07:18:00.745468 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:00.745445 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/84eeb75f-4409-4897-b022-e51ce2d628bf-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t49tn\" (UID: \"84eeb75f-4409-4897-b022-e51ce2d628bf\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t49tn" Jul 17 07:18:00.745566 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:00.745479 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7tg4\" (UniqueName: \"kubernetes.io/projected/84eeb75f-4409-4897-b022-e51ce2d628bf-kube-api-access-s7tg4\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t49tn\" (UID: \"84eeb75f-4409-4897-b022-e51ce2d628bf\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t49tn" Jul 17 07:18:00.745566 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:00.745514 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/84eeb75f-4409-4897-b022-e51ce2d628bf-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t49tn\" (UID: \"84eeb75f-4409-4897-b022-e51ce2d628bf\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t49tn" Jul 17 07:18:00.792015 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:00.791961 2568 generic.go:356] "Generic (PLEG): container finished" podID="eb41a9f6-2e49-48dc-aee9-737d3b981c3e" containerID="e6990eebcf7961cfcb807e10ab0b2d792640023629229c3bf3c23266ef584589" exitCode=0 Jul 17 07:18:00.792114 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:00.792023 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599rfrw" event={"ID":"eb41a9f6-2e49-48dc-aee9-737d3b981c3e","Type":"ContainerDied","Data":"e6990eebcf7961cfcb807e10ab0b2d792640023629229c3bf3c23266ef584589"} Jul 17 07:18:00.792114 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:00.792075 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599rfrw" event={"ID":"eb41a9f6-2e49-48dc-aee9-737d3b981c3e","Type":"ContainerStarted","Data":"4dbd0501d76fd47a34aeddd5498bfcce67aa499dfb215d35dd9ef8c20b944186"} Jul 17 07:18:00.793781 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:00.793759 2568 generic.go:356] "Generic (PLEG): container finished" podID="1448f170-548f-4e5d-a909-d1c97039252a" containerID="1c47f479632b6988225dc113985ad0fccee655c9b0cd52a682b9d694a4178e2f" exitCode=0 Jul 17 07:18:00.793882 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:00.793813 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066lbr" event={"ID":"1448f170-548f-4e5d-a909-d1c97039252a","Type":"ContainerDied","Data":"1c47f479632b6988225dc113985ad0fccee655c9b0cd52a682b9d694a4178e2f"} Jul 17 07:18:00.793882 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:00.793841 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066lbr" event={"ID":"1448f170-548f-4e5d-a909-d1c97039252a","Type":"ContainerStarted","Data":"ca25d89b5910598bc5e41025ec667d3ddfbf0de2d492de4e4985434bbbd0ba0b"} Jul 17 07:18:00.846557 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:00.846532 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/84eeb75f-4409-4897-b022-e51ce2d628bf-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t49tn\" (UID: \"84eeb75f-4409-4897-b022-e51ce2d628bf\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t49tn" Jul 17 07:18:00.846661 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:00.846568 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7tg4\" (UniqueName: \"kubernetes.io/projected/84eeb75f-4409-4897-b022-e51ce2d628bf-kube-api-access-s7tg4\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t49tn\" (UID: \"84eeb75f-4409-4897-b022-e51ce2d628bf\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t49tn" Jul 17 07:18:00.846661 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:00.846609 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/84eeb75f-4409-4897-b022-e51ce2d628bf-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t49tn\" (UID: \"84eeb75f-4409-4897-b022-e51ce2d628bf\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t49tn" Jul 17 07:18:00.846954 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:00.846935 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/84eeb75f-4409-4897-b022-e51ce2d628bf-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t49tn\" (UID: \"84eeb75f-4409-4897-b022-e51ce2d628bf\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t49tn" Jul 17 07:18:00.846954 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:00.846947 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/84eeb75f-4409-4897-b022-e51ce2d628bf-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t49tn\" (UID: \"84eeb75f-4409-4897-b022-e51ce2d628bf\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t49tn" Jul 17 07:18:00.855682 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:00.855647 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7tg4\" (UniqueName: \"kubernetes.io/projected/84eeb75f-4409-4897-b022-e51ce2d628bf-kube-api-access-s7tg4\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t49tn\" (UID: \"84eeb75f-4409-4897-b022-e51ce2d628bf\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t49tn" Jul 17 07:18:00.935981 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:00.935951 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t49tn" Jul 17 07:18:01.016090 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:01.016057 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzvd9"] Jul 17 07:18:01.021073 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:01.021047 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzvd9" Jul 17 07:18:01.034354 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:01.034235 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzvd9"] Jul 17 07:18:01.071264 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:01.071238 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t49tn"] Jul 17 07:18:01.073190 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:18:01.073163 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84eeb75f_4409_4897_b022_e51ce2d628bf.slice/crio-b056bd2f3d7dc8f79ebc3fce3138e29f98e736bd84bf36bed8ce39bcec24421f WatchSource:0}: Error finding container b056bd2f3d7dc8f79ebc3fce3138e29f98e736bd84bf36bed8ce39bcec24421f: Status 404 returned error can't find the container with id b056bd2f3d7dc8f79ebc3fce3138e29f98e736bd84bf36bed8ce39bcec24421f Jul 17 07:18:01.150119 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:01.150088 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1835dd21-d9a4-439e-b224-12ddab9997ed-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzvd9\" (UID: \"1835dd21-d9a4-439e-b224-12ddab9997ed\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzvd9" Jul 17 07:18:01.150260 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:01.150150 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1835dd21-d9a4-439e-b224-12ddab9997ed-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzvd9\" (UID: \"1835dd21-d9a4-439e-b224-12ddab9997ed\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzvd9" Jul 17 07:18:01.150260 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:01.150187 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4g8rs\" (UniqueName: \"kubernetes.io/projected/1835dd21-d9a4-439e-b224-12ddab9997ed-kube-api-access-4g8rs\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzvd9\" (UID: \"1835dd21-d9a4-439e-b224-12ddab9997ed\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzvd9" Jul 17 07:18:01.251048 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:01.251015 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1835dd21-d9a4-439e-b224-12ddab9997ed-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzvd9\" (UID: \"1835dd21-d9a4-439e-b224-12ddab9997ed\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzvd9" Jul 17 07:18:01.251177 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:01.251052 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4g8rs\" (UniqueName: \"kubernetes.io/projected/1835dd21-d9a4-439e-b224-12ddab9997ed-kube-api-access-4g8rs\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzvd9\" (UID: \"1835dd21-d9a4-439e-b224-12ddab9997ed\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzvd9" Jul 17 07:18:01.251177 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:01.251114 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1835dd21-d9a4-439e-b224-12ddab9997ed-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzvd9\" (UID: \"1835dd21-d9a4-439e-b224-12ddab9997ed\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzvd9" Jul 17 07:18:01.251405 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:01.251385 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1835dd21-d9a4-439e-b224-12ddab9997ed-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzvd9\" (UID: \"1835dd21-d9a4-439e-b224-12ddab9997ed\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzvd9" Jul 17 07:18:01.251479 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:01.251460 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1835dd21-d9a4-439e-b224-12ddab9997ed-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzvd9\" (UID: \"1835dd21-d9a4-439e-b224-12ddab9997ed\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzvd9" Jul 17 07:18:01.260988 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:01.260962 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4g8rs\" (UniqueName: \"kubernetes.io/projected/1835dd21-d9a4-439e-b224-12ddab9997ed-kube-api-access-4g8rs\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzvd9\" (UID: \"1835dd21-d9a4-439e-b224-12ddab9997ed\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzvd9" Jul 17 07:18:01.331092 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:01.331027 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzvd9" Jul 17 07:18:01.465082 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:01.465052 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzvd9"] Jul 17 07:18:01.528223 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:18:01.528187 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1835dd21_d9a4_439e_b224_12ddab9997ed.slice/crio-e8a733b7832e3671743fd8ae1980895e32ade1a969d2bb9991aa8db85e154752 WatchSource:0}: Error finding container e8a733b7832e3671743fd8ae1980895e32ade1a969d2bb9991aa8db85e154752: Status 404 returned error can't find the container with id e8a733b7832e3671743fd8ae1980895e32ade1a969d2bb9991aa8db85e154752 Jul 17 07:18:01.800640 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:01.800610 2568 generic.go:356] "Generic (PLEG): container finished" podID="1448f170-548f-4e5d-a909-d1c97039252a" containerID="58cf2463ea0b82ddded20e135e4762e9ba6500c6ea86fd35bac59d96b2f5681b" exitCode=0 Jul 17 07:18:01.801019 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:01.800695 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066lbr" event={"ID":"1448f170-548f-4e5d-a909-d1c97039252a","Type":"ContainerDied","Data":"58cf2463ea0b82ddded20e135e4762e9ba6500c6ea86fd35bac59d96b2f5681b"} Jul 17 07:18:01.802306 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:01.802269 2568 generic.go:356] "Generic (PLEG): container finished" podID="84eeb75f-4409-4897-b022-e51ce2d628bf" containerID="015e9ad39e7abb93caf14ab7c74a78f399912ef7b308c9c5450418394372e845" exitCode=0 Jul 17 07:18:01.802421 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:01.802385 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t49tn" event={"ID":"84eeb75f-4409-4897-b022-e51ce2d628bf","Type":"ContainerDied","Data":"015e9ad39e7abb93caf14ab7c74a78f399912ef7b308c9c5450418394372e845"} Jul 17 07:18:01.802667 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:01.802439 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t49tn" event={"ID":"84eeb75f-4409-4897-b022-e51ce2d628bf","Type":"ContainerStarted","Data":"b056bd2f3d7dc8f79ebc3fce3138e29f98e736bd84bf36bed8ce39bcec24421f"} Jul 17 07:18:01.804133 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:01.804112 2568 generic.go:356] "Generic (PLEG): container finished" podID="1835dd21-d9a4-439e-b224-12ddab9997ed" containerID="58a8cb749a130e40e099e2a21c044a797f309b0eea63d4122f4ef3d743f7e880" exitCode=0 Jul 17 07:18:01.804235 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:01.804197 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzvd9" event={"ID":"1835dd21-d9a4-439e-b224-12ddab9997ed","Type":"ContainerDied","Data":"58a8cb749a130e40e099e2a21c044a797f309b0eea63d4122f4ef3d743f7e880"} Jul 17 07:18:01.804321 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:01.804247 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzvd9" event={"ID":"1835dd21-d9a4-439e-b224-12ddab9997ed","Type":"ContainerStarted","Data":"e8a733b7832e3671743fd8ae1980895e32ade1a969d2bb9991aa8db85e154752"} Jul 17 07:18:01.806931 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:01.806914 2568 generic.go:356] "Generic (PLEG): container finished" podID="eb41a9f6-2e49-48dc-aee9-737d3b981c3e" containerID="46f9738b0d984af01cd85ae5fa2e219b11db1dc8462008e241ff0ec2d19729f6" exitCode=0 Jul 17 07:18:01.807004 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:01.806953 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599rfrw" event={"ID":"eb41a9f6-2e49-48dc-aee9-737d3b981c3e","Type":"ContainerDied","Data":"46f9738b0d984af01cd85ae5fa2e219b11db1dc8462008e241ff0ec2d19729f6"} Jul 17 07:18:02.814619 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:02.814585 2568 generic.go:356] "Generic (PLEG): container finished" podID="eb41a9f6-2e49-48dc-aee9-737d3b981c3e" containerID="1a3fe995346cc709d3c259cec70eb5dd771b9c65805c1f350f29ecbcdb25b3a2" exitCode=0 Jul 17 07:18:02.814939 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:02.814743 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599rfrw" event={"ID":"eb41a9f6-2e49-48dc-aee9-737d3b981c3e","Type":"ContainerDied","Data":"1a3fe995346cc709d3c259cec70eb5dd771b9c65805c1f350f29ecbcdb25b3a2"} Jul 17 07:18:02.816851 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:02.816823 2568 generic.go:356] "Generic (PLEG): container finished" podID="1448f170-548f-4e5d-a909-d1c97039252a" containerID="36cd95bd25c0f96b5a8f6953c8d97fc1fd6c9f33a1da4f909b7dc34ea5217350" exitCode=0 Jul 17 07:18:02.816974 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:02.816875 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066lbr" event={"ID":"1448f170-548f-4e5d-a909-d1c97039252a","Type":"ContainerDied","Data":"36cd95bd25c0f96b5a8f6953c8d97fc1fd6c9f33a1da4f909b7dc34ea5217350"} Jul 17 07:18:03.822786 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:03.822756 2568 generic.go:356] "Generic (PLEG): container finished" podID="1835dd21-d9a4-439e-b224-12ddab9997ed" containerID="cf83cc335f9d479a96351236721a6b8a07056eded8e8b8a71540941201701d1a" exitCode=0 Jul 17 07:18:03.823256 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:03.822844 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzvd9" event={"ID":"1835dd21-d9a4-439e-b224-12ddab9997ed","Type":"ContainerDied","Data":"cf83cc335f9d479a96351236721a6b8a07056eded8e8b8a71540941201701d1a"} Jul 17 07:18:03.824646 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:03.824626 2568 generic.go:356] "Generic (PLEG): container finished" podID="84eeb75f-4409-4897-b022-e51ce2d628bf" containerID="b1bd8237533e609b60941cbe0874d2e3721f9d685fe7bc377ebff71b674c7360" exitCode=0 Jul 17 07:18:03.824734 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:03.824714 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t49tn" event={"ID":"84eeb75f-4409-4897-b022-e51ce2d628bf","Type":"ContainerDied","Data":"b1bd8237533e609b60941cbe0874d2e3721f9d685fe7bc377ebff71b674c7360"} Jul 17 07:18:03.967759 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:03.967690 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599rfrw" Jul 17 07:18:04.004797 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:04.004767 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066lbr" Jul 17 07:18:04.075128 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:04.075065 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eb41a9f6-2e49-48dc-aee9-737d3b981c3e-bundle\") pod \"eb41a9f6-2e49-48dc-aee9-737d3b981c3e\" (UID: \"eb41a9f6-2e49-48dc-aee9-737d3b981c3e\") " Jul 17 07:18:04.075128 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:04.075109 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eb41a9f6-2e49-48dc-aee9-737d3b981c3e-util\") pod \"eb41a9f6-2e49-48dc-aee9-737d3b981c3e\" (UID: \"eb41a9f6-2e49-48dc-aee9-737d3b981c3e\") " Jul 17 07:18:04.075266 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:04.075139 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k7rnd\" (UniqueName: \"kubernetes.io/projected/eb41a9f6-2e49-48dc-aee9-737d3b981c3e-kube-api-access-k7rnd\") pod \"eb41a9f6-2e49-48dc-aee9-737d3b981c3e\" (UID: \"eb41a9f6-2e49-48dc-aee9-737d3b981c3e\") " Jul 17 07:18:04.075588 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:04.075562 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb41a9f6-2e49-48dc-aee9-737d3b981c3e-bundle" (OuterVolumeSpecName: "bundle") pod "eb41a9f6-2e49-48dc-aee9-737d3b981c3e" (UID: "eb41a9f6-2e49-48dc-aee9-737d3b981c3e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:18:04.077050 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:04.077027 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb41a9f6-2e49-48dc-aee9-737d3b981c3e-kube-api-access-k7rnd" (OuterVolumeSpecName: "kube-api-access-k7rnd") pod "eb41a9f6-2e49-48dc-aee9-737d3b981c3e" (UID: "eb41a9f6-2e49-48dc-aee9-737d3b981c3e"). InnerVolumeSpecName "kube-api-access-k7rnd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 07:18:04.079876 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:04.079852 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb41a9f6-2e49-48dc-aee9-737d3b981c3e-util" (OuterVolumeSpecName: "util") pod "eb41a9f6-2e49-48dc-aee9-737d3b981c3e" (UID: "eb41a9f6-2e49-48dc-aee9-737d3b981c3e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:18:04.176016 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:04.175989 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1448f170-548f-4e5d-a909-d1c97039252a-bundle\") pod \"1448f170-548f-4e5d-a909-d1c97039252a\" (UID: \"1448f170-548f-4e5d-a909-d1c97039252a\") " Jul 17 07:18:04.176122 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:04.176028 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1448f170-548f-4e5d-a909-d1c97039252a-util\") pod \"1448f170-548f-4e5d-a909-d1c97039252a\" (UID: \"1448f170-548f-4e5d-a909-d1c97039252a\") " Jul 17 07:18:04.176122 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:04.176060 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vn4db\" (UniqueName: \"kubernetes.io/projected/1448f170-548f-4e5d-a909-d1c97039252a-kube-api-access-vn4db\") pod \"1448f170-548f-4e5d-a909-d1c97039252a\" (UID: \"1448f170-548f-4e5d-a909-d1c97039252a\") " Jul 17 07:18:04.176396 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:04.176364 2568 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eb41a9f6-2e49-48dc-aee9-737d3b981c3e-bundle\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:18:04.176396 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:04.176393 2568 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eb41a9f6-2e49-48dc-aee9-737d3b981c3e-util\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:18:04.176545 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:04.176407 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k7rnd\" (UniqueName: \"kubernetes.io/projected/eb41a9f6-2e49-48dc-aee9-737d3b981c3e-kube-api-access-k7rnd\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:18:04.176545 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:04.176450 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1448f170-548f-4e5d-a909-d1c97039252a-bundle" (OuterVolumeSpecName: "bundle") pod "1448f170-548f-4e5d-a909-d1c97039252a" (UID: "1448f170-548f-4e5d-a909-d1c97039252a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:18:04.178084 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:04.178063 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1448f170-548f-4e5d-a909-d1c97039252a-kube-api-access-vn4db" (OuterVolumeSpecName: "kube-api-access-vn4db") pod "1448f170-548f-4e5d-a909-d1c97039252a" (UID: "1448f170-548f-4e5d-a909-d1c97039252a"). InnerVolumeSpecName "kube-api-access-vn4db". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 07:18:04.182006 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:04.181974 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1448f170-548f-4e5d-a909-d1c97039252a-util" (OuterVolumeSpecName: "util") pod "1448f170-548f-4e5d-a909-d1c97039252a" (UID: "1448f170-548f-4e5d-a909-d1c97039252a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:18:04.277720 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:04.277695 2568 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1448f170-548f-4e5d-a909-d1c97039252a-bundle\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:18:04.277720 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:04.277717 2568 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1448f170-548f-4e5d-a909-d1c97039252a-util\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:18:04.277871 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:04.277726 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vn4db\" (UniqueName: \"kubernetes.io/projected/1448f170-548f-4e5d-a909-d1c97039252a-kube-api-access-vn4db\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:18:04.831559 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:04.831527 2568 generic.go:356] "Generic (PLEG): container finished" podID="1835dd21-d9a4-439e-b224-12ddab9997ed" containerID="ea798229adf38b51fda8d27e14f69ff17c6befed0409647d75b0bff258ab502d" exitCode=0 Jul 17 07:18:04.831931 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:04.831608 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzvd9" event={"ID":"1835dd21-d9a4-439e-b224-12ddab9997ed","Type":"ContainerDied","Data":"ea798229adf38b51fda8d27e14f69ff17c6befed0409647d75b0bff258ab502d"} Jul 17 07:18:04.833425 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:04.833404 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599rfrw" Jul 17 07:18:04.833548 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:04.833424 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599rfrw" event={"ID":"eb41a9f6-2e49-48dc-aee9-737d3b981c3e","Type":"ContainerDied","Data":"4dbd0501d76fd47a34aeddd5498bfcce67aa499dfb215d35dd9ef8c20b944186"} Jul 17 07:18:04.833548 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:04.833464 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4dbd0501d76fd47a34aeddd5498bfcce67aa499dfb215d35dd9ef8c20b944186" Jul 17 07:18:04.836490 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:04.836462 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066lbr" event={"ID":"1448f170-548f-4e5d-a909-d1c97039252a","Type":"ContainerDied","Data":"ca25d89b5910598bc5e41025ec667d3ddfbf0de2d492de4e4985434bbbd0ba0b"} Jul 17 07:18:04.836490 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:04.836484 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066lbr" Jul 17 07:18:04.836636 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:04.836538 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ca25d89b5910598bc5e41025ec667d3ddfbf0de2d492de4e4985434bbbd0ba0b" Jul 17 07:18:04.845488 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:04.845464 2568 generic.go:356] "Generic (PLEG): container finished" podID="84eeb75f-4409-4897-b022-e51ce2d628bf" containerID="fcaf3ed53c18e88d7777a3274e2506aeef24b586ce98a93c5596ec629f41ac8e" exitCode=0 Jul 17 07:18:04.845569 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:04.845519 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t49tn" event={"ID":"84eeb75f-4409-4897-b022-e51ce2d628bf","Type":"ContainerDied","Data":"fcaf3ed53c18e88d7777a3274e2506aeef24b586ce98a93c5596ec629f41ac8e"} Jul 17 07:18:05.984369 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:05.984328 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzvd9" Jul 17 07:18:06.013536 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:06.013514 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t49tn" Jul 17 07:18:06.092881 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:06.092854 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4g8rs\" (UniqueName: \"kubernetes.io/projected/1835dd21-d9a4-439e-b224-12ddab9997ed-kube-api-access-4g8rs\") pod \"1835dd21-d9a4-439e-b224-12ddab9997ed\" (UID: \"1835dd21-d9a4-439e-b224-12ddab9997ed\") " Jul 17 07:18:06.093040 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:06.092888 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1835dd21-d9a4-439e-b224-12ddab9997ed-util\") pod \"1835dd21-d9a4-439e-b224-12ddab9997ed\" (UID: \"1835dd21-d9a4-439e-b224-12ddab9997ed\") " Jul 17 07:18:06.093040 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:06.092918 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1835dd21-d9a4-439e-b224-12ddab9997ed-bundle\") pod \"1835dd21-d9a4-439e-b224-12ddab9997ed\" (UID: \"1835dd21-d9a4-439e-b224-12ddab9997ed\") " Jul 17 07:18:06.093480 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:06.093456 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1835dd21-d9a4-439e-b224-12ddab9997ed-bundle" (OuterVolumeSpecName: "bundle") pod "1835dd21-d9a4-439e-b224-12ddab9997ed" (UID: "1835dd21-d9a4-439e-b224-12ddab9997ed"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:18:06.094966 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:06.094937 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1835dd21-d9a4-439e-b224-12ddab9997ed-kube-api-access-4g8rs" (OuterVolumeSpecName: "kube-api-access-4g8rs") pod "1835dd21-d9a4-439e-b224-12ddab9997ed" (UID: "1835dd21-d9a4-439e-b224-12ddab9997ed"). InnerVolumeSpecName "kube-api-access-4g8rs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 07:18:06.098269 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:06.098249 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1835dd21-d9a4-439e-b224-12ddab9997ed-util" (OuterVolumeSpecName: "util") pod "1835dd21-d9a4-439e-b224-12ddab9997ed" (UID: "1835dd21-d9a4-439e-b224-12ddab9997ed"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:18:06.194132 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:06.194093 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/84eeb75f-4409-4897-b022-e51ce2d628bf-util\") pod \"84eeb75f-4409-4897-b022-e51ce2d628bf\" (UID: \"84eeb75f-4409-4897-b022-e51ce2d628bf\") " Jul 17 07:18:06.194132 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:06.194138 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/84eeb75f-4409-4897-b022-e51ce2d628bf-bundle\") pod \"84eeb75f-4409-4897-b022-e51ce2d628bf\" (UID: \"84eeb75f-4409-4897-b022-e51ce2d628bf\") " Jul 17 07:18:06.194386 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:06.194177 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s7tg4\" (UniqueName: \"kubernetes.io/projected/84eeb75f-4409-4897-b022-e51ce2d628bf-kube-api-access-s7tg4\") pod \"84eeb75f-4409-4897-b022-e51ce2d628bf\" (UID: \"84eeb75f-4409-4897-b022-e51ce2d628bf\") " Jul 17 07:18:06.194386 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:06.194372 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4g8rs\" (UniqueName: \"kubernetes.io/projected/1835dd21-d9a4-439e-b224-12ddab9997ed-kube-api-access-4g8rs\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:18:06.194386 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:06.194384 2568 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1835dd21-d9a4-439e-b224-12ddab9997ed-util\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:18:06.194544 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:06.194394 2568 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1835dd21-d9a4-439e-b224-12ddab9997ed-bundle\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:18:06.194738 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:06.194715 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84eeb75f-4409-4897-b022-e51ce2d628bf-bundle" (OuterVolumeSpecName: "bundle") pod "84eeb75f-4409-4897-b022-e51ce2d628bf" (UID: "84eeb75f-4409-4897-b022-e51ce2d628bf"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:18:06.196233 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:06.196211 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84eeb75f-4409-4897-b022-e51ce2d628bf-kube-api-access-s7tg4" (OuterVolumeSpecName: "kube-api-access-s7tg4") pod "84eeb75f-4409-4897-b022-e51ce2d628bf" (UID: "84eeb75f-4409-4897-b022-e51ce2d628bf"). InnerVolumeSpecName "kube-api-access-s7tg4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 07:18:06.199583 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:06.199559 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84eeb75f-4409-4897-b022-e51ce2d628bf-util" (OuterVolumeSpecName: "util") pod "84eeb75f-4409-4897-b022-e51ce2d628bf" (UID: "84eeb75f-4409-4897-b022-e51ce2d628bf"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:18:06.295226 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:06.295202 2568 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/84eeb75f-4409-4897-b022-e51ce2d628bf-util\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:18:06.295226 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:06.295227 2568 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/84eeb75f-4409-4897-b022-e51ce2d628bf-bundle\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:18:06.295388 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:06.295241 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s7tg4\" (UniqueName: \"kubernetes.io/projected/84eeb75f-4409-4897-b022-e51ce2d628bf-kube-api-access-s7tg4\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:18:06.856479 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:06.856446 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t49tn" Jul 17 07:18:06.856479 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:06.856476 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t49tn" event={"ID":"84eeb75f-4409-4897-b022-e51ce2d628bf","Type":"ContainerDied","Data":"b056bd2f3d7dc8f79ebc3fce3138e29f98e736bd84bf36bed8ce39bcec24421f"} Jul 17 07:18:06.856676 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:06.856523 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b056bd2f3d7dc8f79ebc3fce3138e29f98e736bd84bf36bed8ce39bcec24421f" Jul 17 07:18:06.858411 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:06.858389 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzvd9" event={"ID":"1835dd21-d9a4-439e-b224-12ddab9997ed","Type":"ContainerDied","Data":"e8a733b7832e3671743fd8ae1980895e32ade1a969d2bb9991aa8db85e154752"} Jul 17 07:18:06.858545 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:06.858423 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzvd9" Jul 17 07:18:06.858545 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:06.858427 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e8a733b7832e3671743fd8ae1980895e32ade1a969d2bb9991aa8db85e154752" Jul 17 07:18:17.472603 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:17.472562 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sqb8k"] Jul 17 07:18:17.472966 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:17.472916 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1448f170-548f-4e5d-a909-d1c97039252a" containerName="extract" Jul 17 07:18:17.472966 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:17.472928 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="1448f170-548f-4e5d-a909-d1c97039252a" containerName="extract" Jul 17 07:18:17.472966 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:17.472939 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1835dd21-d9a4-439e-b224-12ddab9997ed" containerName="util" Jul 17 07:18:17.472966 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:17.472946 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="1835dd21-d9a4-439e-b224-12ddab9997ed" containerName="util" Jul 17 07:18:17.472966 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:17.472960 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84eeb75f-4409-4897-b022-e51ce2d628bf" containerName="pull" Jul 17 07:18:17.472966 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:17.472966 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="84eeb75f-4409-4897-b022-e51ce2d628bf" containerName="pull" Jul 17 07:18:17.473148 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:17.472975 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb41a9f6-2e49-48dc-aee9-737d3b981c3e" containerName="pull" Jul 17 07:18:17.473148 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:17.472980 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb41a9f6-2e49-48dc-aee9-737d3b981c3e" containerName="pull" Jul 17 07:18:17.473148 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:17.472986 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1835dd21-d9a4-439e-b224-12ddab9997ed" containerName="extract" Jul 17 07:18:17.473148 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:17.472991 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="1835dd21-d9a4-439e-b224-12ddab9997ed" containerName="extract" Jul 17 07:18:17.473148 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:17.472997 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84eeb75f-4409-4897-b022-e51ce2d628bf" containerName="extract" Jul 17 07:18:17.473148 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:17.473002 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="84eeb75f-4409-4897-b022-e51ce2d628bf" containerName="extract" Jul 17 07:18:17.473148 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:17.473009 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1448f170-548f-4e5d-a909-d1c97039252a" containerName="util" Jul 17 07:18:17.473148 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:17.473013 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="1448f170-548f-4e5d-a909-d1c97039252a" containerName="util" Jul 17 07:18:17.473148 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:17.473019 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1448f170-548f-4e5d-a909-d1c97039252a" containerName="pull" Jul 17 07:18:17.473148 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:17.473025 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="1448f170-548f-4e5d-a909-d1c97039252a" containerName="pull" Jul 17 07:18:17.473148 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:17.473033 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1835dd21-d9a4-439e-b224-12ddab9997ed" containerName="pull" Jul 17 07:18:17.473148 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:17.473041 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="1835dd21-d9a4-439e-b224-12ddab9997ed" containerName="pull" Jul 17 07:18:17.473148 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:17.473048 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84eeb75f-4409-4897-b022-e51ce2d628bf" containerName="util" Jul 17 07:18:17.473148 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:17.473053 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="84eeb75f-4409-4897-b022-e51ce2d628bf" containerName="util" Jul 17 07:18:17.473148 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:17.473058 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb41a9f6-2e49-48dc-aee9-737d3b981c3e" containerName="extract" Jul 17 07:18:17.473148 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:17.473063 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb41a9f6-2e49-48dc-aee9-737d3b981c3e" containerName="extract" Jul 17 07:18:17.473148 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:17.473069 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb41a9f6-2e49-48dc-aee9-737d3b981c3e" containerName="util" Jul 17 07:18:17.473148 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:17.473075 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb41a9f6-2e49-48dc-aee9-737d3b981c3e" containerName="util" Jul 17 07:18:17.473148 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:17.473132 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="84eeb75f-4409-4897-b022-e51ce2d628bf" containerName="extract" Jul 17 07:18:17.473148 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:17.473140 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="1835dd21-d9a4-439e-b224-12ddab9997ed" containerName="extract" Jul 17 07:18:17.473148 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:17.473147 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="1448f170-548f-4e5d-a909-d1c97039252a" containerName="extract" Jul 17 07:18:17.473148 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:17.473155 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb41a9f6-2e49-48dc-aee9-737d3b981c3e" containerName="extract" Jul 17 07:18:17.481464 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:17.481446 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sqb8k" Jul 17 07:18:17.484117 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:17.484091 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-pvj5d\"" Jul 17 07:18:17.495064 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:17.495042 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sqb8k"] Jul 17 07:18:17.575262 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:17.575228 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e4075f50-d03d-400a-9e12-777ce16c7d35-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sqb8k\" (UID: \"e4075f50-d03d-400a-9e12-777ce16c7d35\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sqb8k" Jul 17 07:18:17.575430 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:17.575328 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnwbq\" (UniqueName: \"kubernetes.io/projected/e4075f50-d03d-400a-9e12-777ce16c7d35-kube-api-access-pnwbq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sqb8k\" (UID: \"e4075f50-d03d-400a-9e12-777ce16c7d35\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sqb8k" Jul 17 07:18:17.676025 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:17.675993 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e4075f50-d03d-400a-9e12-777ce16c7d35-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sqb8k\" (UID: \"e4075f50-d03d-400a-9e12-777ce16c7d35\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sqb8k" Jul 17 07:18:17.676198 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:17.676079 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnwbq\" (UniqueName: \"kubernetes.io/projected/e4075f50-d03d-400a-9e12-777ce16c7d35-kube-api-access-pnwbq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sqb8k\" (UID: \"e4075f50-d03d-400a-9e12-777ce16c7d35\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sqb8k" Jul 17 07:18:17.676429 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:17.676409 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e4075f50-d03d-400a-9e12-777ce16c7d35-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sqb8k\" (UID: \"e4075f50-d03d-400a-9e12-777ce16c7d35\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sqb8k" Jul 17 07:18:17.693886 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:17.693853 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnwbq\" (UniqueName: \"kubernetes.io/projected/e4075f50-d03d-400a-9e12-777ce16c7d35-kube-api-access-pnwbq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sqb8k\" (UID: \"e4075f50-d03d-400a-9e12-777ce16c7d35\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sqb8k" Jul 17 07:18:17.790699 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:17.790626 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sqb8k" Jul 17 07:18:17.933624 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:17.933485 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sqb8k"] Jul 17 07:18:17.936974 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:18:17.936912 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4075f50_d03d_400a_9e12_777ce16c7d35.slice/crio-0d4eb8a3257affc025ce583e8cf0e966471e821a23b2b6674bcc0996ae7585ab WatchSource:0}: Error finding container 0d4eb8a3257affc025ce583e8cf0e966471e821a23b2b6674bcc0996ae7585ab: Status 404 returned error can't find the container with id 0d4eb8a3257affc025ce583e8cf0e966471e821a23b2b6674bcc0996ae7585ab Jul 17 07:18:18.917879 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:18.917839 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sqb8k" event={"ID":"e4075f50-d03d-400a-9e12-777ce16c7d35","Type":"ContainerStarted","Data":"0d4eb8a3257affc025ce583e8cf0e966471e821a23b2b6674bcc0996ae7585ab"} Jul 17 07:18:20.076635 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:20.076594 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-nt9qn"] Jul 17 07:18:20.080266 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:20.080239 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-nt9qn" Jul 17 07:18:20.083319 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:20.083279 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-s6tdj\"" Jul 17 07:18:20.095666 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:20.095645 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-nt9qn"] Jul 17 07:18:20.098590 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:20.098564 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xl2l\" (UniqueName: \"kubernetes.io/projected/1789e5ec-2d38-4951-90b9-62c28eea5c43-kube-api-access-5xl2l\") pod \"authorino-operator-75cc8d5cb7-nt9qn\" (UID: \"1789e5ec-2d38-4951-90b9-62c28eea5c43\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-nt9qn" Jul 17 07:18:20.199543 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:20.199505 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xl2l\" (UniqueName: \"kubernetes.io/projected/1789e5ec-2d38-4951-90b9-62c28eea5c43-kube-api-access-5xl2l\") pod \"authorino-operator-75cc8d5cb7-nt9qn\" (UID: \"1789e5ec-2d38-4951-90b9-62c28eea5c43\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-nt9qn" Jul 17 07:18:20.208209 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:20.208187 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xl2l\" (UniqueName: \"kubernetes.io/projected/1789e5ec-2d38-4951-90b9-62c28eea5c43-kube-api-access-5xl2l\") pod \"authorino-operator-75cc8d5cb7-nt9qn\" (UID: \"1789e5ec-2d38-4951-90b9-62c28eea5c43\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-nt9qn" Jul 17 07:18:20.393231 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:20.393202 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-nt9qn" Jul 17 07:18:20.548165 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:20.545967 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-nt9qn"] Jul 17 07:18:20.548973 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:18:20.548780 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1789e5ec_2d38_4951_90b9_62c28eea5c43.slice/crio-e710b0e878481de64c150f584365e8d9d4b7762e279e7c03ad85fa0bb02ac544 WatchSource:0}: Error finding container e710b0e878481de64c150f584365e8d9d4b7762e279e7c03ad85fa0bb02ac544: Status 404 returned error can't find the container with id e710b0e878481de64c150f584365e8d9d4b7762e279e7c03ad85fa0bb02ac544 Jul 17 07:18:20.932060 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:20.932016 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-nt9qn" event={"ID":"1789e5ec-2d38-4951-90b9-62c28eea5c43","Type":"ContainerStarted","Data":"e710b0e878481de64c150f584365e8d9d4b7762e279e7c03ad85fa0bb02ac544"} Jul 17 07:18:22.572403 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:22.572369 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-djvpx"] Jul 17 07:18:22.576143 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:22.576120 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-djvpx" Jul 17 07:18:22.578786 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:22.578759 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 17 07:18:22.579043 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:22.579024 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-m6dqw\"" Jul 17 07:18:22.588874 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:22.588851 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-djvpx"] Jul 17 07:18:22.618356 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:22.618319 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77qpq\" (UniqueName: \"kubernetes.io/projected/c60a3dfd-f974-44b5-b128-6bfe009dd784-kube-api-access-77qpq\") pod \"dns-operator-controller-manager-864dd78c4c-djvpx\" (UID: \"c60a3dfd-f974-44b5-b128-6bfe009dd784\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-djvpx" Jul 17 07:18:22.718793 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:22.718750 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77qpq\" (UniqueName: \"kubernetes.io/projected/c60a3dfd-f974-44b5-b128-6bfe009dd784-kube-api-access-77qpq\") pod \"dns-operator-controller-manager-864dd78c4c-djvpx\" (UID: \"c60a3dfd-f974-44b5-b128-6bfe009dd784\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-djvpx" Jul 17 07:18:22.730175 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:22.730143 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77qpq\" (UniqueName: \"kubernetes.io/projected/c60a3dfd-f974-44b5-b128-6bfe009dd784-kube-api-access-77qpq\") pod \"dns-operator-controller-manager-864dd78c4c-djvpx\" (UID: \"c60a3dfd-f974-44b5-b128-6bfe009dd784\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-djvpx" Jul 17 07:18:22.888443 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:22.888409 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-djvpx" Jul 17 07:18:24.188296 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:24.188254 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-djvpx"] Jul 17 07:18:24.189672 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:18:24.189643 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc60a3dfd_f974_44b5_b128_6bfe009dd784.slice/crio-5a81cecd8ea3145e089e455eda1a0b58a3de32ae1e21d96f4812438ff0d33b77 WatchSource:0}: Error finding container 5a81cecd8ea3145e089e455eda1a0b58a3de32ae1e21d96f4812438ff0d33b77: Status 404 returned error can't find the container with id 5a81cecd8ea3145e089e455eda1a0b58a3de32ae1e21d96f4812438ff0d33b77 Jul 17 07:18:24.415014 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:24.414977 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p668g"] Jul 17 07:18:24.418533 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:24.418516 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p668g" Jul 17 07:18:24.421606 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:24.421588 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-bk7fv\"" Jul 17 07:18:24.431816 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:24.431796 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcrjr\" (UniqueName: \"kubernetes.io/projected/3d2a1be8-7bb6-47be-bea8-2595e7ef261e-kube-api-access-gcrjr\") pod \"limitador-operator-controller-manager-859d7b78bd-p668g\" (UID: \"3d2a1be8-7bb6-47be-bea8-2595e7ef261e\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p668g" Jul 17 07:18:24.440211 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:24.440186 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p668g"] Jul 17 07:18:24.532737 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:24.532704 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcrjr\" (UniqueName: \"kubernetes.io/projected/3d2a1be8-7bb6-47be-bea8-2595e7ef261e-kube-api-access-gcrjr\") pod \"limitador-operator-controller-manager-859d7b78bd-p668g\" (UID: \"3d2a1be8-7bb6-47be-bea8-2595e7ef261e\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p668g" Jul 17 07:18:24.543023 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:24.543000 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcrjr\" (UniqueName: \"kubernetes.io/projected/3d2a1be8-7bb6-47be-bea8-2595e7ef261e-kube-api-access-gcrjr\") pod \"limitador-operator-controller-manager-859d7b78bd-p668g\" (UID: \"3d2a1be8-7bb6-47be-bea8-2595e7ef261e\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p668g" Jul 17 07:18:24.728488 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:24.728409 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p668g" Jul 17 07:18:24.869040 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:24.869009 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p668g"] Jul 17 07:18:24.880976 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:18:24.880926 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d2a1be8_7bb6_47be_bea8_2595e7ef261e.slice/crio-1f0cb4176dae17da2f85da171aad7efd0952964864ac43edea7c7ae713cb534d WatchSource:0}: Error finding container 1f0cb4176dae17da2f85da171aad7efd0952964864ac43edea7c7ae713cb534d: Status 404 returned error can't find the container with id 1f0cb4176dae17da2f85da171aad7efd0952964864ac43edea7c7ae713cb534d Jul 17 07:18:24.958318 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:24.958252 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sqb8k" event={"ID":"e4075f50-d03d-400a-9e12-777ce16c7d35","Type":"ContainerStarted","Data":"13c4a84fd3d525d96fa9a61e082e349a11ea982f1fdac89d66100db1377f10ca"} Jul 17 07:18:24.958491 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:24.958476 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sqb8k" Jul 17 07:18:24.960356 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:24.960274 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-nt9qn" event={"ID":"1789e5ec-2d38-4951-90b9-62c28eea5c43","Type":"ContainerStarted","Data":"3348aab20ec585d3fa805d020fc63672ebe4373ea56c12d0a822f789ddf6a31b"} Jul 17 07:18:24.960356 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:24.960360 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-nt9qn" Jul 17 07:18:24.961855 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:24.961801 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p668g" event={"ID":"3d2a1be8-7bb6-47be-bea8-2595e7ef261e","Type":"ContainerStarted","Data":"1f0cb4176dae17da2f85da171aad7efd0952964864ac43edea7c7ae713cb534d"} Jul 17 07:18:24.963152 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:24.963115 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-djvpx" event={"ID":"c60a3dfd-f974-44b5-b128-6bfe009dd784","Type":"ContainerStarted","Data":"5a81cecd8ea3145e089e455eda1a0b58a3de32ae1e21d96f4812438ff0d33b77"} Jul 17 07:18:24.997381 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:24.997202 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sqb8k" podStartSLOduration=1.8781617019999999 podStartE2EDuration="7.99718794s" totalImagesPullingTime="6.119026238s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 07:18:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:18:24.994722363 +0000 UTC m=+409.479411339" watchObservedRunningTime="2026-07-17 07:18:24.99718794 +0000 UTC m=+409.481876913" Jul 17 07:18:25.025573 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:25.025520 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-nt9qn" podStartSLOduration=1.5319334580000001 podStartE2EDuration="5.025498986s" totalImagesPullingTime="3.493565528s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 07:18:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:18:25.02536992 +0000 UTC m=+409.510058894" watchObservedRunningTime="2026-07-17 07:18:25.025498986 +0000 UTC m=+409.510187961" Jul 17 07:18:26.976809 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:26.976720 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-djvpx" event={"ID":"c60a3dfd-f974-44b5-b128-6bfe009dd784","Type":"ContainerStarted","Data":"19511fc1686b991b65dea62621ee6df8bfb70d30875cb834a5718b2270a966f0"} Jul 17 07:18:26.977220 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:26.976888 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-djvpx" Jul 17 07:18:27.000298 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:27.000244 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-djvpx" podStartSLOduration=2.510765954 podStartE2EDuration="5.000229868s" totalImagesPullingTime="2.489463914s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 07:18:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:18:26.99641795 +0000 UTC m=+411.481106924" watchObservedRunningTime="2026-07-17 07:18:27.000229868 +0000 UTC m=+411.484918862" Jul 17 07:18:27.103889 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:27.103857 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-hmtld"] Jul 17 07:18:27.106974 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:27.106952 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hmtld" Jul 17 07:18:27.109457 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:27.109433 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 17 07:18:27.109598 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:27.109433 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 17 07:18:27.109687 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:27.109671 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-vqgqw\"" Jul 17 07:18:27.135302 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:27.120188 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-hmtld"] Jul 17 07:18:27.151937 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:27.151907 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/be85768d-f7f1-4da5-b8e8-7a58e8c8f9bc-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-hmtld\" (UID: \"be85768d-f7f1-4da5-b8e8-7a58e8c8f9bc\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hmtld" Jul 17 07:18:27.152113 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:27.151969 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/be85768d-f7f1-4da5-b8e8-7a58e8c8f9bc-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-hmtld\" (UID: \"be85768d-f7f1-4da5-b8e8-7a58e8c8f9bc\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hmtld" Jul 17 07:18:27.152113 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:27.151998 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wm6d\" (UniqueName: \"kubernetes.io/projected/be85768d-f7f1-4da5-b8e8-7a58e8c8f9bc-kube-api-access-7wm6d\") pod \"kuadrant-console-plugin-66b5774985-hmtld\" (UID: \"be85768d-f7f1-4da5-b8e8-7a58e8c8f9bc\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hmtld" Jul 17 07:18:27.252547 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:27.252486 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/be85768d-f7f1-4da5-b8e8-7a58e8c8f9bc-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-hmtld\" (UID: \"be85768d-f7f1-4da5-b8e8-7a58e8c8f9bc\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hmtld" Jul 17 07:18:27.252547 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:27.252524 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/be85768d-f7f1-4da5-b8e8-7a58e8c8f9bc-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-hmtld\" (UID: \"be85768d-f7f1-4da5-b8e8-7a58e8c8f9bc\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hmtld" Jul 17 07:18:27.252718 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:27.252548 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wm6d\" (UniqueName: \"kubernetes.io/projected/be85768d-f7f1-4da5-b8e8-7a58e8c8f9bc-kube-api-access-7wm6d\") pod \"kuadrant-console-plugin-66b5774985-hmtld\" (UID: \"be85768d-f7f1-4da5-b8e8-7a58e8c8f9bc\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hmtld" Jul 17 07:18:27.252718 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:18:27.252647 2568 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 17 07:18:27.252718 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:18:27.252706 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be85768d-f7f1-4da5-b8e8-7a58e8c8f9bc-plugin-serving-cert podName:be85768d-f7f1-4da5-b8e8-7a58e8c8f9bc nodeName:}" failed. No retries permitted until 2026-07-17 07:18:27.752687346 +0000 UTC m=+412.237376305 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/be85768d-f7f1-4da5-b8e8-7a58e8c8f9bc-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-hmtld" (UID: "be85768d-f7f1-4da5-b8e8-7a58e8c8f9bc") : secret "plugin-serving-cert" not found Jul 17 07:18:27.253152 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:27.253128 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/be85768d-f7f1-4da5-b8e8-7a58e8c8f9bc-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-hmtld\" (UID: \"be85768d-f7f1-4da5-b8e8-7a58e8c8f9bc\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hmtld" Jul 17 07:18:27.268157 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:27.268132 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wm6d\" (UniqueName: \"kubernetes.io/projected/be85768d-f7f1-4da5-b8e8-7a58e8c8f9bc-kube-api-access-7wm6d\") pod \"kuadrant-console-plugin-66b5774985-hmtld\" (UID: \"be85768d-f7f1-4da5-b8e8-7a58e8c8f9bc\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hmtld" Jul 17 07:18:27.755627 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:27.755585 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/be85768d-f7f1-4da5-b8e8-7a58e8c8f9bc-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-hmtld\" (UID: \"be85768d-f7f1-4da5-b8e8-7a58e8c8f9bc\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hmtld" Jul 17 07:18:27.758009 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:27.757984 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/be85768d-f7f1-4da5-b8e8-7a58e8c8f9bc-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-hmtld\" (UID: \"be85768d-f7f1-4da5-b8e8-7a58e8c8f9bc\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hmtld" Jul 17 07:18:27.988858 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:27.988815 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p668g" event={"ID":"3d2a1be8-7bb6-47be-bea8-2595e7ef261e","Type":"ContainerStarted","Data":"484c51fb6898e3bedb12d2642aca2c22bd95ba1b0ba371c3e54663a1654b39b5"} Jul 17 07:18:27.989311 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:27.988910 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p668g" Jul 17 07:18:28.009077 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:28.008987 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p668g" podStartSLOduration=1.745548331 podStartE2EDuration="4.008972501s" totalImagesPullingTime="2.26342417s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 07:18:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:18:28.007048684 +0000 UTC m=+412.491737649" watchObservedRunningTime="2026-07-17 07:18:28.008972501 +0000 UTC m=+412.493661475" Jul 17 07:18:28.016209 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:28.016186 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hmtld" Jul 17 07:18:28.153403 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:28.153376 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-hmtld"] Jul 17 07:18:28.154951 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:18:28.154924 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbe85768d_f7f1_4da5_b8e8_7a58e8c8f9bc.slice/crio-736379b417977a64483d61aaba6695c3e1791d970063b41729941178f0e492a4 WatchSource:0}: Error finding container 736379b417977a64483d61aaba6695c3e1791d970063b41729941178f0e492a4: Status 404 returned error can't find the container with id 736379b417977a64483d61aaba6695c3e1791d970063b41729941178f0e492a4 Jul 17 07:18:28.996554 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:28.996520 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hmtld" event={"ID":"be85768d-f7f1-4da5-b8e8-7a58e8c8f9bc","Type":"ContainerStarted","Data":"736379b417977a64483d61aaba6695c3e1791d970063b41729941178f0e492a4"} Jul 17 07:18:35.971143 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:35.971110 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sqb8k" Jul 17 07:18:35.971493 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:35.971203 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-nt9qn" Jul 17 07:18:37.780361 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:37.780317 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sqb8k"] Jul 17 07:18:37.780820 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:37.780566 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sqb8k" podUID="e4075f50-d03d-400a-9e12-777ce16c7d35" containerName="manager" containerID="cri-o://13c4a84fd3d525d96fa9a61e082e349a11ea982f1fdac89d66100db1377f10ca" gracePeriod=2 Jul 17 07:18:37.782643 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:18:37.782600 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-sqb8k\" is forbidden: User \"system:node:ip-10-0-138-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-233.ec2.internal' and this object" podUID="e4075f50-d03d-400a-9e12-777ce16c7d35" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sqb8k" Jul 17 07:18:37.783356 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:37.783336 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sqb8k"] Jul 17 07:18:37.801767 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:37.801741 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p668g"] Jul 17 07:18:37.802021 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:37.801990 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p668g" podUID="3d2a1be8-7bb6-47be-bea8-2595e7ef261e" containerName="manager" containerID="cri-o://484c51fb6898e3bedb12d2642aca2c22bd95ba1b0ba371c3e54663a1654b39b5" gracePeriod=2 Jul 17 07:18:37.804738 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:37.804715 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p668g" Jul 17 07:18:37.809607 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:37.809439 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xphtj"] Jul 17 07:18:37.809931 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:37.809911 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4075f50-d03d-400a-9e12-777ce16c7d35" containerName="manager" Jul 17 07:18:37.810063 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:37.809936 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4075f50-d03d-400a-9e12-777ce16c7d35" containerName="manager" Jul 17 07:18:37.810268 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:37.810249 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4075f50-d03d-400a-9e12-777ce16c7d35" containerName="manager" Jul 17 07:18:37.813431 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:37.813410 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p668g"] Jul 17 07:18:37.813567 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:37.813502 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xphtj" Jul 17 07:18:37.813891 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:18:37.813497 2568 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"3d2a1be8-7bb6-47be-bea8-2595e7ef261e\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-p668g\": pods \"limitador-operator-controller-manager-859d7b78bd-p668g\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p668g" Jul 17 07:18:37.815827 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:18:37.815797 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-sqb8k\" is forbidden: User \"system:node:ip-10-0-138-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-233.ec2.internal' and this object" podUID="e4075f50-d03d-400a-9e12-777ce16c7d35" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sqb8k" Jul 17 07:18:37.817729 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:18:37.817700 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-p668g\" is forbidden: User \"system:node:ip-10-0-138-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-233.ec2.internal' and this object" podUID="3d2a1be8-7bb6-47be-bea8-2595e7ef261e" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p668g" Jul 17 07:18:37.827223 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:37.827196 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bqvbq"] Jul 17 07:18:37.828435 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:37.827684 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d2a1be8-7bb6-47be-bea8-2595e7ef261e" containerName="manager" Jul 17 07:18:37.828435 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:37.827706 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d2a1be8-7bb6-47be-bea8-2595e7ef261e" containerName="manager" Jul 17 07:18:37.828435 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:37.827793 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d2a1be8-7bb6-47be-bea8-2595e7ef261e" containerName="manager" Jul 17 07:18:37.843494 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:37.842576 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bqvbq" Jul 17 07:18:37.845721 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:37.845629 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xphtj"] Jul 17 07:18:37.847440 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:37.847414 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bqvbq"] Jul 17 07:18:37.848225 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:18:37.847987 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-sqb8k\" is forbidden: User \"system:node:ip-10-0-138-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-233.ec2.internal' and this object" podUID="e4075f50-d03d-400a-9e12-777ce16c7d35" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sqb8k" Jul 17 07:18:37.848375 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:37.848266 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5xgc\" (UniqueName: \"kubernetes.io/projected/2e5405de-8236-4737-bb75-86f07fc3e538-kube-api-access-t5xgc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xphtj\" (UID: \"2e5405de-8236-4737-bb75-86f07fc3e538\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xphtj" Jul 17 07:18:37.848457 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:37.848387 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2e5405de-8236-4737-bb75-86f07fc3e538-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xphtj\" (UID: \"2e5405de-8236-4737-bb75-86f07fc3e538\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xphtj" Jul 17 07:18:37.849805 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:18:37.849767 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-p668g\" is forbidden: User \"system:node:ip-10-0-138-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-233.ec2.internal' and this object" podUID="3d2a1be8-7bb6-47be-bea8-2595e7ef261e" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p668g" Jul 17 07:18:37.855010 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:18:37.854982 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-sqb8k\" is forbidden: User \"system:node:ip-10-0-138-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-233.ec2.internal' and this object" podUID="e4075f50-d03d-400a-9e12-777ce16c7d35" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sqb8k" Jul 17 07:18:37.949508 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:37.949469 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5xgc\" (UniqueName: \"kubernetes.io/projected/2e5405de-8236-4737-bb75-86f07fc3e538-kube-api-access-t5xgc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xphtj\" (UID: \"2e5405de-8236-4737-bb75-86f07fc3e538\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xphtj" Jul 17 07:18:37.949508 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:37.949513 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsjj9\" (UniqueName: \"kubernetes.io/projected/b26606a2-fa29-424a-bc05-f350e61cc291-kube-api-access-rsjj9\") pod \"limitador-operator-controller-manager-859d7b78bd-bqvbq\" (UID: \"b26606a2-fa29-424a-bc05-f350e61cc291\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bqvbq" Jul 17 07:18:37.949735 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:37.949553 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2e5405de-8236-4737-bb75-86f07fc3e538-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xphtj\" (UID: \"2e5405de-8236-4737-bb75-86f07fc3e538\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xphtj" Jul 17 07:18:37.949933 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:37.949908 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2e5405de-8236-4737-bb75-86f07fc3e538-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xphtj\" (UID: \"2e5405de-8236-4737-bb75-86f07fc3e538\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xphtj" Jul 17 07:18:37.959239 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:37.959216 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5xgc\" (UniqueName: \"kubernetes.io/projected/2e5405de-8236-4737-bb75-86f07fc3e538-kube-api-access-t5xgc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xphtj\" (UID: \"2e5405de-8236-4737-bb75-86f07fc3e538\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xphtj" Jul 17 07:18:37.992680 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:37.992650 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-djvpx" Jul 17 07:18:38.011685 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:18:38.011645 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-p668g\" is forbidden: User \"system:node:ip-10-0-138-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-233.ec2.internal' and this object" podUID="3d2a1be8-7bb6-47be-bea8-2595e7ef261e" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p668g" Jul 17 07:18:38.013606 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:18:38.013574 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-sqb8k\" is forbidden: User \"system:node:ip-10-0-138-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-233.ec2.internal' and this object" podUID="e4075f50-d03d-400a-9e12-777ce16c7d35" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sqb8k" Jul 17 07:18:38.050303 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:38.050198 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsjj9\" (UniqueName: \"kubernetes.io/projected/b26606a2-fa29-424a-bc05-f350e61cc291-kube-api-access-rsjj9\") pod \"limitador-operator-controller-manager-859d7b78bd-bqvbq\" (UID: \"b26606a2-fa29-424a-bc05-f350e61cc291\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bqvbq" Jul 17 07:18:38.062251 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:38.062223 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsjj9\" (UniqueName: \"kubernetes.io/projected/b26606a2-fa29-424a-bc05-f350e61cc291-kube-api-access-rsjj9\") pod \"limitador-operator-controller-manager-859d7b78bd-bqvbq\" (UID: \"b26606a2-fa29-424a-bc05-f350e61cc291\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bqvbq" Jul 17 07:18:38.202895 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:38.202863 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xphtj" Jul 17 07:18:38.209913 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:38.209886 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bqvbq" Jul 17 07:18:51.447263 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:51.447186 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bqvbq"] Jul 17 07:18:51.450479 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:51.450458 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p668g" Jul 17 07:18:51.452947 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:18:51.452908 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb26606a2_fa29_424a_bc05_f350e61cc291.slice/crio-5ec4a5ef7950ad866d241c61b6eb832d293377c2c1b8bf6e6495b5b7627985ee WatchSource:0}: Error finding container 5ec4a5ef7950ad866d241c61b6eb832d293377c2c1b8bf6e6495b5b7627985ee: Status 404 returned error can't find the container with id 5ec4a5ef7950ad866d241c61b6eb832d293377c2c1b8bf6e6495b5b7627985ee Jul 17 07:18:51.453061 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:18:51.453038 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-p668g\" is forbidden: User \"system:node:ip-10-0-138-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-233.ec2.internal' and this object" podUID="3d2a1be8-7bb6-47be-bea8-2595e7ef261e" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p668g" Jul 17 07:18:51.459903 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:51.459888 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sqb8k" Jul 17 07:18:51.461919 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:18:51.461891 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-p668g\" is forbidden: User \"system:node:ip-10-0-138-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-233.ec2.internal' and this object" podUID="3d2a1be8-7bb6-47be-bea8-2595e7ef261e" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p668g" Jul 17 07:18:51.463871 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:18:51.463850 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-sqb8k\" is forbidden: User \"system:node:ip-10-0-138-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-233.ec2.internal' and this object" podUID="e4075f50-d03d-400a-9e12-777ce16c7d35" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sqb8k" Jul 17 07:18:51.472244 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:51.472221 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xphtj"] Jul 17 07:18:51.474726 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:18:51.474699 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e5405de_8236_4737_bb75_86f07fc3e538.slice/crio-5b1e655d9d0ab91c25cbd3f58d1d6ab39c9bd39248e2b094cc7d364936e524eb WatchSource:0}: Error finding container 5b1e655d9d0ab91c25cbd3f58d1d6ab39c9bd39248e2b094cc7d364936e524eb: Status 404 returned error can't find the container with id 5b1e655d9d0ab91c25cbd3f58d1d6ab39c9bd39248e2b094cc7d364936e524eb Jul 17 07:18:51.575956 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:51.575851 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e4075f50-d03d-400a-9e12-777ce16c7d35-extensions-socket-volume\") pod \"e4075f50-d03d-400a-9e12-777ce16c7d35\" (UID: \"e4075f50-d03d-400a-9e12-777ce16c7d35\") " Jul 17 07:18:51.576124 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:51.575991 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pnwbq\" (UniqueName: \"kubernetes.io/projected/e4075f50-d03d-400a-9e12-777ce16c7d35-kube-api-access-pnwbq\") pod \"e4075f50-d03d-400a-9e12-777ce16c7d35\" (UID: \"e4075f50-d03d-400a-9e12-777ce16c7d35\") " Jul 17 07:18:51.576124 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:51.576018 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gcrjr\" (UniqueName: \"kubernetes.io/projected/3d2a1be8-7bb6-47be-bea8-2595e7ef261e-kube-api-access-gcrjr\") pod \"3d2a1be8-7bb6-47be-bea8-2595e7ef261e\" (UID: \"3d2a1be8-7bb6-47be-bea8-2595e7ef261e\") " Jul 17 07:18:51.576359 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:51.576326 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4075f50-d03d-400a-9e12-777ce16c7d35-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e4075f50-d03d-400a-9e12-777ce16c7d35" (UID: "e4075f50-d03d-400a-9e12-777ce16c7d35"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:18:51.578270 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:51.578238 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d2a1be8-7bb6-47be-bea8-2595e7ef261e-kube-api-access-gcrjr" (OuterVolumeSpecName: "kube-api-access-gcrjr") pod "3d2a1be8-7bb6-47be-bea8-2595e7ef261e" (UID: "3d2a1be8-7bb6-47be-bea8-2595e7ef261e"). InnerVolumeSpecName "kube-api-access-gcrjr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 07:18:51.578270 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:51.578249 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4075f50-d03d-400a-9e12-777ce16c7d35-kube-api-access-pnwbq" (OuterVolumeSpecName: "kube-api-access-pnwbq") pod "e4075f50-d03d-400a-9e12-777ce16c7d35" (UID: "e4075f50-d03d-400a-9e12-777ce16c7d35"). InnerVolumeSpecName "kube-api-access-pnwbq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 07:18:51.676983 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:51.676947 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pnwbq\" (UniqueName: \"kubernetes.io/projected/e4075f50-d03d-400a-9e12-777ce16c7d35-kube-api-access-pnwbq\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:18:51.676983 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:51.676978 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gcrjr\" (UniqueName: \"kubernetes.io/projected/3d2a1be8-7bb6-47be-bea8-2595e7ef261e-kube-api-access-gcrjr\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:18:51.676983 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:51.676987 2568 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e4075f50-d03d-400a-9e12-777ce16c7d35-extensions-socket-volume\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:18:51.999618 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:51.999577 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d2a1be8-7bb6-47be-bea8-2595e7ef261e" path="/var/lib/kubelet/pods/3d2a1be8-7bb6-47be-bea8-2595e7ef261e/volumes" Jul 17 07:18:52.000023 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:52.000004 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4075f50-d03d-400a-9e12-777ce16c7d35" path="/var/lib/kubelet/pods/e4075f50-d03d-400a-9e12-777ce16c7d35/volumes" Jul 17 07:18:52.134707 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:52.134673 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xphtj" event={"ID":"2e5405de-8236-4737-bb75-86f07fc3e538","Type":"ContainerStarted","Data":"fa4bacbcfe0e88975f9bce080bd49c52f3919066ec24ceb8e9414b2b1fdabf85"} Jul 17 07:18:52.134887 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:52.134727 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xphtj" event={"ID":"2e5405de-8236-4737-bb75-86f07fc3e538","Type":"ContainerStarted","Data":"5b1e655d9d0ab91c25cbd3f58d1d6ab39c9bd39248e2b094cc7d364936e524eb"} Jul 17 07:18:52.134887 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:52.134781 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xphtj" Jul 17 07:18:52.136037 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:52.136014 2568 generic.go:356] "Generic (PLEG): container finished" podID="3d2a1be8-7bb6-47be-bea8-2595e7ef261e" containerID="484c51fb6898e3bedb12d2642aca2c22bd95ba1b0ba371c3e54663a1654b39b5" exitCode=0 Jul 17 07:18:52.136168 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:52.136061 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p668g" Jul 17 07:18:52.136168 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:52.136107 2568 scope.go:117] "RemoveContainer" containerID="484c51fb6898e3bedb12d2642aca2c22bd95ba1b0ba371c3e54663a1654b39b5" Jul 17 07:18:52.137593 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:52.137569 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hmtld" event={"ID":"be85768d-f7f1-4da5-b8e8-7a58e8c8f9bc","Type":"ContainerStarted","Data":"20390b1d72098553160127865969c60d1efacd94d734272ca731208efd2c1080"} Jul 17 07:18:52.139601 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:52.139580 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bqvbq" event={"ID":"b26606a2-fa29-424a-bc05-f350e61cc291","Type":"ContainerStarted","Data":"e73c1ae16215ad72b74bbe5db0113e36db8881cc4e6d48d5aeae7ae829e6238a"} Jul 17 07:18:52.139708 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:52.139616 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bqvbq" event={"ID":"b26606a2-fa29-424a-bc05-f350e61cc291","Type":"ContainerStarted","Data":"5ec4a5ef7950ad866d241c61b6eb832d293377c2c1b8bf6e6495b5b7627985ee"} Jul 17 07:18:52.139708 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:52.139674 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bqvbq" Jul 17 07:18:52.140903 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:52.140884 2568 generic.go:356] "Generic (PLEG): container finished" podID="e4075f50-d03d-400a-9e12-777ce16c7d35" containerID="13c4a84fd3d525d96fa9a61e082e349a11ea982f1fdac89d66100db1377f10ca" exitCode=0 Jul 17 07:18:52.140976 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:52.140955 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sqb8k" Jul 17 07:18:52.145710 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:52.145684 2568 scope.go:117] "RemoveContainer" containerID="484c51fb6898e3bedb12d2642aca2c22bd95ba1b0ba371c3e54663a1654b39b5" Jul 17 07:18:52.145955 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:18:52.145938 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"484c51fb6898e3bedb12d2642aca2c22bd95ba1b0ba371c3e54663a1654b39b5\": container with ID starting with 484c51fb6898e3bedb12d2642aca2c22bd95ba1b0ba371c3e54663a1654b39b5 not found: ID does not exist" containerID="484c51fb6898e3bedb12d2642aca2c22bd95ba1b0ba371c3e54663a1654b39b5" Jul 17 07:18:52.146012 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:52.145965 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"484c51fb6898e3bedb12d2642aca2c22bd95ba1b0ba371c3e54663a1654b39b5"} err="failed to get container status \"484c51fb6898e3bedb12d2642aca2c22bd95ba1b0ba371c3e54663a1654b39b5\": rpc error: code = NotFound desc = could not find container \"484c51fb6898e3bedb12d2642aca2c22bd95ba1b0ba371c3e54663a1654b39b5\": container with ID starting with 484c51fb6898e3bedb12d2642aca2c22bd95ba1b0ba371c3e54663a1654b39b5 not found: ID does not exist" Jul 17 07:18:52.146012 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:52.145981 2568 scope.go:117] "RemoveContainer" containerID="13c4a84fd3d525d96fa9a61e082e349a11ea982f1fdac89d66100db1377f10ca" Jul 17 07:18:52.153454 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:52.153433 2568 scope.go:117] "RemoveContainer" containerID="13c4a84fd3d525d96fa9a61e082e349a11ea982f1fdac89d66100db1377f10ca" Jul 17 07:18:52.153759 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:18:52.153735 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13c4a84fd3d525d96fa9a61e082e349a11ea982f1fdac89d66100db1377f10ca\": container with ID starting with 13c4a84fd3d525d96fa9a61e082e349a11ea982f1fdac89d66100db1377f10ca not found: ID does not exist" containerID="13c4a84fd3d525d96fa9a61e082e349a11ea982f1fdac89d66100db1377f10ca" Jul 17 07:18:52.153824 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:52.153765 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13c4a84fd3d525d96fa9a61e082e349a11ea982f1fdac89d66100db1377f10ca"} err="failed to get container status \"13c4a84fd3d525d96fa9a61e082e349a11ea982f1fdac89d66100db1377f10ca\": rpc error: code = NotFound desc = could not find container \"13c4a84fd3d525d96fa9a61e082e349a11ea982f1fdac89d66100db1377f10ca\": container with ID starting with 13c4a84fd3d525d96fa9a61e082e349a11ea982f1fdac89d66100db1377f10ca not found: ID does not exist" Jul 17 07:18:52.155274 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:52.155235 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xphtj" podStartSLOduration=15.15522374 podStartE2EDuration="15.15522374s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 07:18:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:18:52.154524656 +0000 UTC m=+436.639213659" watchObservedRunningTime="2026-07-17 07:18:52.15522374 +0000 UTC m=+436.639912713" Jul 17 07:18:52.156570 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:18:52.156545 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-p668g\" is forbidden: User \"system:node:ip-10-0-138-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-233.ec2.internal' and this object" podUID="3d2a1be8-7bb6-47be-bea8-2595e7ef261e" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p668g" Jul 17 07:18:52.158389 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:18:52.158362 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-sqb8k\" is forbidden: User \"system:node:ip-10-0-138-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-233.ec2.internal' and this object" podUID="e4075f50-d03d-400a-9e12-777ce16c7d35" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sqb8k" Jul 17 07:18:52.177485 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:52.177441 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hmtld" podStartSLOduration=1.946074327 podStartE2EDuration="25.177426334s" totalImagesPullingTime="23.231352007s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 07:18:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:18:52.175634965 +0000 UTC m=+436.660323941" watchObservedRunningTime="2026-07-17 07:18:52.177426334 +0000 UTC m=+436.662115309" Jul 17 07:18:52.193752 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:52.193705 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bqvbq" podStartSLOduration=15.193690065 podStartE2EDuration="15.193690065s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 07:18:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:18:52.192475434 +0000 UTC m=+436.677164401" watchObservedRunningTime="2026-07-17 07:18:52.193690065 +0000 UTC m=+436.678379038" Jul 17 07:18:55.838914 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:55.838879 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xphtj"] Jul 17 07:18:55.839343 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:55.839060 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xphtj" podUID="2e5405de-8236-4737-bb75-86f07fc3e538" containerName="manager" containerID="cri-o://fa4bacbcfe0e88975f9bce080bd49c52f3919066ec24ceb8e9414b2b1fdabf85" gracePeriod=10 Jul 17 07:18:56.115418 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:56.115387 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xphtj" Jul 17 07:18:56.166594 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:56.166562 2568 generic.go:356] "Generic (PLEG): container finished" podID="2e5405de-8236-4737-bb75-86f07fc3e538" containerID="fa4bacbcfe0e88975f9bce080bd49c52f3919066ec24ceb8e9414b2b1fdabf85" exitCode=0 Jul 17 07:18:56.166735 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:56.166611 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xphtj" Jul 17 07:18:56.166735 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:56.166661 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xphtj" event={"ID":"2e5405de-8236-4737-bb75-86f07fc3e538","Type":"ContainerDied","Data":"fa4bacbcfe0e88975f9bce080bd49c52f3919066ec24ceb8e9414b2b1fdabf85"} Jul 17 07:18:56.166815 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:56.166743 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xphtj" event={"ID":"2e5405de-8236-4737-bb75-86f07fc3e538","Type":"ContainerDied","Data":"5b1e655d9d0ab91c25cbd3f58d1d6ab39c9bd39248e2b094cc7d364936e524eb"} Jul 17 07:18:56.166815 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:56.166772 2568 scope.go:117] "RemoveContainer" containerID="fa4bacbcfe0e88975f9bce080bd49c52f3919066ec24ceb8e9414b2b1fdabf85" Jul 17 07:18:56.175110 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:56.175093 2568 scope.go:117] "RemoveContainer" containerID="fa4bacbcfe0e88975f9bce080bd49c52f3919066ec24ceb8e9414b2b1fdabf85" Jul 17 07:18:56.175383 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:18:56.175360 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa4bacbcfe0e88975f9bce080bd49c52f3919066ec24ceb8e9414b2b1fdabf85\": container with ID starting with fa4bacbcfe0e88975f9bce080bd49c52f3919066ec24ceb8e9414b2b1fdabf85 not found: ID does not exist" containerID="fa4bacbcfe0e88975f9bce080bd49c52f3919066ec24ceb8e9414b2b1fdabf85" Jul 17 07:18:56.175437 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:56.175396 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa4bacbcfe0e88975f9bce080bd49c52f3919066ec24ceb8e9414b2b1fdabf85"} err="failed to get container status \"fa4bacbcfe0e88975f9bce080bd49c52f3919066ec24ceb8e9414b2b1fdabf85\": rpc error: code = NotFound desc = could not find container \"fa4bacbcfe0e88975f9bce080bd49c52f3919066ec24ceb8e9414b2b1fdabf85\": container with ID starting with fa4bacbcfe0e88975f9bce080bd49c52f3919066ec24ceb8e9414b2b1fdabf85 not found: ID does not exist" Jul 17 07:18:56.216265 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:56.216230 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t5xgc\" (UniqueName: \"kubernetes.io/projected/2e5405de-8236-4737-bb75-86f07fc3e538-kube-api-access-t5xgc\") pod \"2e5405de-8236-4737-bb75-86f07fc3e538\" (UID: \"2e5405de-8236-4737-bb75-86f07fc3e538\") " Jul 17 07:18:56.216430 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:56.216356 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2e5405de-8236-4737-bb75-86f07fc3e538-extensions-socket-volume\") pod \"2e5405de-8236-4737-bb75-86f07fc3e538\" (UID: \"2e5405de-8236-4737-bb75-86f07fc3e538\") " Jul 17 07:18:56.216718 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:56.216695 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e5405de-8236-4737-bb75-86f07fc3e538-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "2e5405de-8236-4737-bb75-86f07fc3e538" (UID: "2e5405de-8236-4737-bb75-86f07fc3e538"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:18:56.218188 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:56.218170 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e5405de-8236-4737-bb75-86f07fc3e538-kube-api-access-t5xgc" (OuterVolumeSpecName: "kube-api-access-t5xgc") pod "2e5405de-8236-4737-bb75-86f07fc3e538" (UID: "2e5405de-8236-4737-bb75-86f07fc3e538"). InnerVolumeSpecName "kube-api-access-t5xgc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 07:18:56.317459 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:56.317424 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t5xgc\" (UniqueName: \"kubernetes.io/projected/2e5405de-8236-4737-bb75-86f07fc3e538-kube-api-access-t5xgc\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:18:56.317459 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:56.317452 2568 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2e5405de-8236-4737-bb75-86f07fc3e538-extensions-socket-volume\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:18:56.496432 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:56.496400 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xphtj"] Jul 17 07:18:56.500501 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:56.500472 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xphtj"] Jul 17 07:18:57.999757 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:18:57.999723 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e5405de-8236-4737-bb75-86f07fc3e538" path="/var/lib/kubelet/pods/2e5405de-8236-4737-bb75-86f07fc3e538/volumes" Jul 17 07:19:03.150842 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:03.150810 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bqvbq" Jul 17 07:19:12.271955 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:12.271915 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zd7fg"] Jul 17 07:19:12.272633 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:12.272610 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e5405de-8236-4737-bb75-86f07fc3e538" containerName="manager" Jul 17 07:19:12.272633 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:12.272633 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e5405de-8236-4737-bb75-86f07fc3e538" containerName="manager" Jul 17 07:19:12.272849 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:12.272833 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e5405de-8236-4737-bb75-86f07fc3e538" containerName="manager" Jul 17 07:19:12.502809 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:12.502777 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zd7fg"] Jul 17 07:19:12.502967 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:12.502880 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zd7fg" Jul 17 07:19:12.506110 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:12.506091 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-4vnl6\"" Jul 17 07:19:12.655562 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:12.655530 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7852377-14b6-4b77-98d3-bc5546fdebe5-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-zd7fg\" (UID: \"c7852377-14b6-4b77-98d3-bc5546fdebe5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zd7fg" Jul 17 07:19:12.655731 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:12.655571 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7852377-14b6-4b77-98d3-bc5546fdebe5-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-zd7fg\" (UID: \"c7852377-14b6-4b77-98d3-bc5546fdebe5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zd7fg" Jul 17 07:19:12.655731 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:12.655594 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7852377-14b6-4b77-98d3-bc5546fdebe5-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-zd7fg\" (UID: \"c7852377-14b6-4b77-98d3-bc5546fdebe5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zd7fg" Jul 17 07:19:12.655731 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:12.655632 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7852377-14b6-4b77-98d3-bc5546fdebe5-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-zd7fg\" (UID: \"c7852377-14b6-4b77-98d3-bc5546fdebe5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zd7fg" Jul 17 07:19:12.655731 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:12.655681 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7852377-14b6-4b77-98d3-bc5546fdebe5-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-zd7fg\" (UID: \"c7852377-14b6-4b77-98d3-bc5546fdebe5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zd7fg" Jul 17 07:19:12.655731 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:12.655706 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7852377-14b6-4b77-98d3-bc5546fdebe5-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-zd7fg\" (UID: \"c7852377-14b6-4b77-98d3-bc5546fdebe5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zd7fg" Jul 17 07:19:12.655731 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:12.655722 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d99hh\" (UniqueName: \"kubernetes.io/projected/c7852377-14b6-4b77-98d3-bc5546fdebe5-kube-api-access-d99hh\") pod \"maas-default-gateway-openshift-default-56654dcf7-zd7fg\" (UID: \"c7852377-14b6-4b77-98d3-bc5546fdebe5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zd7fg" Jul 17 07:19:12.655942 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:12.655745 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7852377-14b6-4b77-98d3-bc5546fdebe5-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-zd7fg\" (UID: \"c7852377-14b6-4b77-98d3-bc5546fdebe5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zd7fg" Jul 17 07:19:12.655942 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:12.655811 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7852377-14b6-4b77-98d3-bc5546fdebe5-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-zd7fg\" (UID: \"c7852377-14b6-4b77-98d3-bc5546fdebe5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zd7fg" Jul 17 07:19:12.756667 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:12.756628 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7852377-14b6-4b77-98d3-bc5546fdebe5-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-zd7fg\" (UID: \"c7852377-14b6-4b77-98d3-bc5546fdebe5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zd7fg" Jul 17 07:19:12.756837 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:12.756675 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7852377-14b6-4b77-98d3-bc5546fdebe5-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-zd7fg\" (UID: \"c7852377-14b6-4b77-98d3-bc5546fdebe5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zd7fg" Jul 17 07:19:12.756837 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:12.756703 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7852377-14b6-4b77-98d3-bc5546fdebe5-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-zd7fg\" (UID: \"c7852377-14b6-4b77-98d3-bc5546fdebe5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zd7fg" Jul 17 07:19:12.756837 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:12.756734 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7852377-14b6-4b77-98d3-bc5546fdebe5-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-zd7fg\" (UID: \"c7852377-14b6-4b77-98d3-bc5546fdebe5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zd7fg" Jul 17 07:19:12.756837 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:12.756775 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7852377-14b6-4b77-98d3-bc5546fdebe5-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-zd7fg\" (UID: \"c7852377-14b6-4b77-98d3-bc5546fdebe5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zd7fg" Jul 17 07:19:12.756837 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:12.756810 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7852377-14b6-4b77-98d3-bc5546fdebe5-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-zd7fg\" (UID: \"c7852377-14b6-4b77-98d3-bc5546fdebe5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zd7fg" Jul 17 07:19:12.756837 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:12.756835 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d99hh\" (UniqueName: \"kubernetes.io/projected/c7852377-14b6-4b77-98d3-bc5546fdebe5-kube-api-access-d99hh\") pod \"maas-default-gateway-openshift-default-56654dcf7-zd7fg\" (UID: \"c7852377-14b6-4b77-98d3-bc5546fdebe5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zd7fg" Jul 17 07:19:12.757148 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:12.756870 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7852377-14b6-4b77-98d3-bc5546fdebe5-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-zd7fg\" (UID: \"c7852377-14b6-4b77-98d3-bc5546fdebe5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zd7fg" Jul 17 07:19:12.757148 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:12.756921 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7852377-14b6-4b77-98d3-bc5546fdebe5-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-zd7fg\" (UID: \"c7852377-14b6-4b77-98d3-bc5546fdebe5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zd7fg" Jul 17 07:19:12.757148 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:12.757133 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7852377-14b6-4b77-98d3-bc5546fdebe5-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-zd7fg\" (UID: \"c7852377-14b6-4b77-98d3-bc5546fdebe5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zd7fg" Jul 17 07:19:12.757314 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:12.757211 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7852377-14b6-4b77-98d3-bc5546fdebe5-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-zd7fg\" (UID: \"c7852377-14b6-4b77-98d3-bc5546fdebe5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zd7fg" Jul 17 07:19:12.757314 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:12.757241 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7852377-14b6-4b77-98d3-bc5546fdebe5-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-zd7fg\" (UID: \"c7852377-14b6-4b77-98d3-bc5546fdebe5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zd7fg" Jul 17 07:19:12.757433 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:12.757335 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7852377-14b6-4b77-98d3-bc5546fdebe5-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-zd7fg\" (UID: \"c7852377-14b6-4b77-98d3-bc5546fdebe5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zd7fg" Jul 17 07:19:12.757539 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:12.757521 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7852377-14b6-4b77-98d3-bc5546fdebe5-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-zd7fg\" (UID: \"c7852377-14b6-4b77-98d3-bc5546fdebe5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zd7fg" Jul 17 07:19:12.759033 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:12.759004 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7852377-14b6-4b77-98d3-bc5546fdebe5-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-zd7fg\" (UID: \"c7852377-14b6-4b77-98d3-bc5546fdebe5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zd7fg" Jul 17 07:19:12.759240 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:12.759216 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7852377-14b6-4b77-98d3-bc5546fdebe5-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-zd7fg\" (UID: \"c7852377-14b6-4b77-98d3-bc5546fdebe5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zd7fg" Jul 17 07:19:12.766276 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:12.766251 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7852377-14b6-4b77-98d3-bc5546fdebe5-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-zd7fg\" (UID: \"c7852377-14b6-4b77-98d3-bc5546fdebe5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zd7fg" Jul 17 07:19:12.766481 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:12.766383 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d99hh\" (UniqueName: \"kubernetes.io/projected/c7852377-14b6-4b77-98d3-bc5546fdebe5-kube-api-access-d99hh\") pod \"maas-default-gateway-openshift-default-56654dcf7-zd7fg\" (UID: \"c7852377-14b6-4b77-98d3-bc5546fdebe5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zd7fg" Jul 17 07:19:12.813336 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:12.813313 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zd7fg" Jul 17 07:19:12.990823 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:12.990793 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zd7fg"] Jul 17 07:19:12.991770 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:19:12.991745 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7852377_14b6_4b77_98d3_bc5546fdebe5.slice/crio-244c2038f478e097b6a6304bbb2fb25e5b9e267c5875556ac6c4e1906c9ac735 WatchSource:0}: Error finding container 244c2038f478e097b6a6304bbb2fb25e5b9e267c5875556ac6c4e1906c9ac735: Status 404 returned error can't find the container with id 244c2038f478e097b6a6304bbb2fb25e5b9e267c5875556ac6c4e1906c9ac735 Jul 17 07:19:12.993983 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:12.993948 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:19:12.994054 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:12.994014 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:19:12.994054 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:12.994040 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:19:13.261343 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:13.261242 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zd7fg" event={"ID":"c7852377-14b6-4b77-98d3-bc5546fdebe5","Type":"ContainerStarted","Data":"4c2e8a642f829d82025a66c8e3f7f19f51bb76fd10bfcae4eaba50e8005023d7"} Jul 17 07:19:13.261343 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:13.261331 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zd7fg" event={"ID":"c7852377-14b6-4b77-98d3-bc5546fdebe5","Type":"ContainerStarted","Data":"244c2038f478e097b6a6304bbb2fb25e5b9e267c5875556ac6c4e1906c9ac735"} Jul 17 07:19:13.282707 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:13.282662 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zd7fg" podStartSLOduration=1.282646341 podStartE2EDuration="1.282646341s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 07:19:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:19:13.281860249 +0000 UTC m=+457.766549222" watchObservedRunningTime="2026-07-17 07:19:13.282646341 +0000 UTC m=+457.767335314" Jul 17 07:19:13.813643 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:13.813612 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zd7fg" Jul 17 07:19:14.384662 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:14.384625 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-r24kq"] Jul 17 07:19:14.807370 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:14.807266 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-r24kq"] Jul 17 07:19:14.807370 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:14.807339 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-r24kq"] Jul 17 07:19:14.807539 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:14.807421 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-r24kq" Jul 17 07:19:14.809741 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:14.809720 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 17 07:19:14.818978 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:14.818958 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zd7fg" Jul 17 07:19:14.977980 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:14.977946 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xv55\" (UniqueName: \"kubernetes.io/projected/97870d03-571c-4ea4-a322-7f082f436ea5-kube-api-access-4xv55\") pod \"limitador-limitador-64c94fd64f-r24kq\" (UID: \"97870d03-571c-4ea4-a322-7f082f436ea5\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-r24kq" Jul 17 07:19:14.977980 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:14.977991 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/97870d03-571c-4ea4-a322-7f082f436ea5-config-file\") pod \"limitador-limitador-64c94fd64f-r24kq\" (UID: \"97870d03-571c-4ea4-a322-7f082f436ea5\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-r24kq" Jul 17 07:19:15.079306 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:15.079209 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/97870d03-571c-4ea4-a322-7f082f436ea5-config-file\") pod \"limitador-limitador-64c94fd64f-r24kq\" (UID: \"97870d03-571c-4ea4-a322-7f082f436ea5\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-r24kq" Jul 17 07:19:15.079461 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:15.079325 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xv55\" (UniqueName: \"kubernetes.io/projected/97870d03-571c-4ea4-a322-7f082f436ea5-kube-api-access-4xv55\") pod \"limitador-limitador-64c94fd64f-r24kq\" (UID: \"97870d03-571c-4ea4-a322-7f082f436ea5\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-r24kq" Jul 17 07:19:15.079845 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:15.079824 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/97870d03-571c-4ea4-a322-7f082f436ea5-config-file\") pod \"limitador-limitador-64c94fd64f-r24kq\" (UID: \"97870d03-571c-4ea4-a322-7f082f436ea5\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-r24kq" Jul 17 07:19:15.091261 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:15.091239 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xv55\" (UniqueName: \"kubernetes.io/projected/97870d03-571c-4ea4-a322-7f082f436ea5-kube-api-access-4xv55\") pod \"limitador-limitador-64c94fd64f-r24kq\" (UID: \"97870d03-571c-4ea4-a322-7f082f436ea5\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-r24kq" Jul 17 07:19:15.118745 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:15.118719 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-r24kq" Jul 17 07:19:15.184171 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:15.184128 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-cknt6"] Jul 17 07:19:15.277769 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:15.277738 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-cknt6"] Jul 17 07:19:15.278003 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:15.277977 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zd7fg" Jul 17 07:19:15.278113 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:15.278097 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zd7fg" Jul 17 07:19:15.278163 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:15.278009 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-cknt6" Jul 17 07:19:15.280605 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:15.280581 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-2kklx\"" Jul 17 07:19:15.382519 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:15.382487 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkt46\" (UniqueName: \"kubernetes.io/projected/c1b06560-ac99-4177-8775-a4f1750ed64c-kube-api-access-tkt46\") pod \"authorino-58cd955675-cknt6\" (UID: \"c1b06560-ac99-4177-8775-a4f1750ed64c\") " pod="kuadrant-system/authorino-58cd955675-cknt6" Jul 17 07:19:15.407169 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:15.407141 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-r24kq"] Jul 17 07:19:15.408099 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:19:15.408068 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97870d03_571c_4ea4_a322_7f082f436ea5.slice/crio-9f6f8c4d47a0461abc6f25c7b21581d12f64eca53f4b9945d676ce9af8a04fc5 WatchSource:0}: Error finding container 9f6f8c4d47a0461abc6f25c7b21581d12f64eca53f4b9945d676ce9af8a04fc5: Status 404 returned error can't find the container with id 9f6f8c4d47a0461abc6f25c7b21581d12f64eca53f4b9945d676ce9af8a04fc5 Jul 17 07:19:15.483982 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:15.483950 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkt46\" (UniqueName: \"kubernetes.io/projected/c1b06560-ac99-4177-8775-a4f1750ed64c-kube-api-access-tkt46\") pod \"authorino-58cd955675-cknt6\" (UID: \"c1b06560-ac99-4177-8775-a4f1750ed64c\") " pod="kuadrant-system/authorino-58cd955675-cknt6" Jul 17 07:19:15.492195 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:15.492174 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkt46\" (UniqueName: \"kubernetes.io/projected/c1b06560-ac99-4177-8775-a4f1750ed64c-kube-api-access-tkt46\") pod \"authorino-58cd955675-cknt6\" (UID: \"c1b06560-ac99-4177-8775-a4f1750ed64c\") " pod="kuadrant-system/authorino-58cd955675-cknt6" Jul 17 07:19:15.593386 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:15.593356 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-cknt6" Jul 17 07:19:15.762498 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:15.762470 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-cknt6"] Jul 17 07:19:15.764246 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:19:15.764213 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc1b06560_ac99_4177_8775_a4f1750ed64c.slice/crio-a3207e2d8826da1ee127954551a0f9178bd87d0aa5c1ba674e40fa6a5fdc3612 WatchSource:0}: Error finding container a3207e2d8826da1ee127954551a0f9178bd87d0aa5c1ba674e40fa6a5fdc3612: Status 404 returned error can't find the container with id a3207e2d8826da1ee127954551a0f9178bd87d0aa5c1ba674e40fa6a5fdc3612 Jul 17 07:19:16.284957 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:16.284868 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-cknt6" event={"ID":"c1b06560-ac99-4177-8775-a4f1750ed64c","Type":"ContainerStarted","Data":"a3207e2d8826da1ee127954551a0f9178bd87d0aa5c1ba674e40fa6a5fdc3612"} Jul 17 07:19:16.287546 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:16.287493 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-r24kq" event={"ID":"97870d03-571c-4ea4-a322-7f082f436ea5","Type":"ContainerStarted","Data":"9f6f8c4d47a0461abc6f25c7b21581d12f64eca53f4b9945d676ce9af8a04fc5"} Jul 17 07:19:20.000379 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:20.000278 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-cknt6"] Jul 17 07:19:20.315658 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:20.315560 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-r24kq" event={"ID":"97870d03-571c-4ea4-a322-7f082f436ea5","Type":"ContainerStarted","Data":"20cf51c7b30d0eca81138411f5f6db2d91c74ce12f5173205b513f6f13165518"} Jul 17 07:19:20.315827 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:20.315728 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-r24kq" Jul 17 07:19:20.317073 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:20.317044 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-cknt6" event={"ID":"c1b06560-ac99-4177-8775-a4f1750ed64c","Type":"ContainerStarted","Data":"4ef381a48a66c03b81a71b06122eefd76abb9faf92e92e7e9ff536edb6fa7698"} Jul 17 07:19:20.334051 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:20.334009 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-r24kq" podStartSLOduration=2.027623182 podStartE2EDuration="6.33399631s" totalImagesPullingTime="4.306373128s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 07:19:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:19:20.332411707 +0000 UTC m=+464.817100682" watchObservedRunningTime="2026-07-17 07:19:20.33399631 +0000 UTC m=+464.818685280" Jul 17 07:19:20.348904 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:20.348861 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-cknt6" podStartSLOduration=1.396725761 podStartE2EDuration="5.348849532s" totalImagesPullingTime="3.952123771s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 07:19:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:19:20.347380421 +0000 UTC m=+464.832069396" watchObservedRunningTime="2026-07-17 07:19:20.348849532 +0000 UTC m=+464.833538505" Jul 17 07:19:21.323211 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:21.323168 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-cknt6" podUID="c1b06560-ac99-4177-8775-a4f1750ed64c" containerName="authorino" containerID="cri-o://4ef381a48a66c03b81a71b06122eefd76abb9faf92e92e7e9ff536edb6fa7698" gracePeriod=30 Jul 17 07:19:21.596688 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:21.596629 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-cknt6" Jul 17 07:19:21.634188 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:21.634159 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tkt46\" (UniqueName: \"kubernetes.io/projected/c1b06560-ac99-4177-8775-a4f1750ed64c-kube-api-access-tkt46\") pod \"c1b06560-ac99-4177-8775-a4f1750ed64c\" (UID: \"c1b06560-ac99-4177-8775-a4f1750ed64c\") " Jul 17 07:19:21.636491 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:21.636456 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1b06560-ac99-4177-8775-a4f1750ed64c-kube-api-access-tkt46" (OuterVolumeSpecName: "kube-api-access-tkt46") pod "c1b06560-ac99-4177-8775-a4f1750ed64c" (UID: "c1b06560-ac99-4177-8775-a4f1750ed64c"). InnerVolumeSpecName "kube-api-access-tkt46". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 07:19:21.735537 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:21.735506 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tkt46\" (UniqueName: \"kubernetes.io/projected/c1b06560-ac99-4177-8775-a4f1750ed64c-kube-api-access-tkt46\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:19:22.330185 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:22.330099 2568 generic.go:356] "Generic (PLEG): container finished" podID="c1b06560-ac99-4177-8775-a4f1750ed64c" containerID="4ef381a48a66c03b81a71b06122eefd76abb9faf92e92e7e9ff536edb6fa7698" exitCode=0 Jul 17 07:19:22.330185 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:22.330142 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-cknt6" Jul 17 07:19:22.330185 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:22.330159 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-cknt6" event={"ID":"c1b06560-ac99-4177-8775-a4f1750ed64c","Type":"ContainerDied","Data":"4ef381a48a66c03b81a71b06122eefd76abb9faf92e92e7e9ff536edb6fa7698"} Jul 17 07:19:22.330732 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:22.330199 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-cknt6" event={"ID":"c1b06560-ac99-4177-8775-a4f1750ed64c","Type":"ContainerDied","Data":"a3207e2d8826da1ee127954551a0f9178bd87d0aa5c1ba674e40fa6a5fdc3612"} Jul 17 07:19:22.330732 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:22.330222 2568 scope.go:117] "RemoveContainer" containerID="4ef381a48a66c03b81a71b06122eefd76abb9faf92e92e7e9ff536edb6fa7698" Jul 17 07:19:22.339601 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:22.339582 2568 scope.go:117] "RemoveContainer" containerID="4ef381a48a66c03b81a71b06122eefd76abb9faf92e92e7e9ff536edb6fa7698" Jul 17 07:19:22.339877 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:19:22.339858 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ef381a48a66c03b81a71b06122eefd76abb9faf92e92e7e9ff536edb6fa7698\": container with ID starting with 4ef381a48a66c03b81a71b06122eefd76abb9faf92e92e7e9ff536edb6fa7698 not found: ID does not exist" containerID="4ef381a48a66c03b81a71b06122eefd76abb9faf92e92e7e9ff536edb6fa7698" Jul 17 07:19:22.339921 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:22.339887 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ef381a48a66c03b81a71b06122eefd76abb9faf92e92e7e9ff536edb6fa7698"} err="failed to get container status \"4ef381a48a66c03b81a71b06122eefd76abb9faf92e92e7e9ff536edb6fa7698\": rpc error: code = NotFound desc = could not find container \"4ef381a48a66c03b81a71b06122eefd76abb9faf92e92e7e9ff536edb6fa7698\": container with ID starting with 4ef381a48a66c03b81a71b06122eefd76abb9faf92e92e7e9ff536edb6fa7698 not found: ID does not exist" Jul 17 07:19:22.347922 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:22.347898 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-cknt6"] Jul 17 07:19:22.352042 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:22.352021 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-cknt6"] Jul 17 07:19:23.999843 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:23.999812 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1b06560-ac99-4177-8775-a4f1750ed64c" path="/var/lib/kubelet/pods/c1b06560-ac99-4177-8775-a4f1750ed64c/volumes" Jul 17 07:19:29.246769 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:29.246736 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-r24kq"] Jul 17 07:19:29.247395 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:29.246961 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-r24kq" podUID="97870d03-571c-4ea4-a322-7f082f436ea5" containerName="limitador" containerID="cri-o://20cf51c7b30d0eca81138411f5f6db2d91c74ce12f5173205b513f6f13165518" gracePeriod=30 Jul 17 07:19:29.247829 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:29.247804 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-r24kq" Jul 17 07:19:29.801785 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:29.801763 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-r24kq" Jul 17 07:19:29.898423 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:29.898396 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4xv55\" (UniqueName: \"kubernetes.io/projected/97870d03-571c-4ea4-a322-7f082f436ea5-kube-api-access-4xv55\") pod \"97870d03-571c-4ea4-a322-7f082f436ea5\" (UID: \"97870d03-571c-4ea4-a322-7f082f436ea5\") " Jul 17 07:19:29.898570 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:29.898439 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/97870d03-571c-4ea4-a322-7f082f436ea5-config-file\") pod \"97870d03-571c-4ea4-a322-7f082f436ea5\" (UID: \"97870d03-571c-4ea4-a322-7f082f436ea5\") " Jul 17 07:19:29.898806 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:29.898784 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97870d03-571c-4ea4-a322-7f082f436ea5-config-file" (OuterVolumeSpecName: "config-file") pod "97870d03-571c-4ea4-a322-7f082f436ea5" (UID: "97870d03-571c-4ea4-a322-7f082f436ea5"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 07:19:29.900389 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:29.900362 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97870d03-571c-4ea4-a322-7f082f436ea5-kube-api-access-4xv55" (OuterVolumeSpecName: "kube-api-access-4xv55") pod "97870d03-571c-4ea4-a322-7f082f436ea5" (UID: "97870d03-571c-4ea4-a322-7f082f436ea5"). InnerVolumeSpecName "kube-api-access-4xv55". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 07:19:29.999844 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:29.999815 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4xv55\" (UniqueName: \"kubernetes.io/projected/97870d03-571c-4ea4-a322-7f082f436ea5-kube-api-access-4xv55\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:19:29.999844 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:29.999841 2568 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/97870d03-571c-4ea4-a322-7f082f436ea5-config-file\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:19:30.380898 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:30.380866 2568 generic.go:356] "Generic (PLEG): container finished" podID="97870d03-571c-4ea4-a322-7f082f436ea5" containerID="20cf51c7b30d0eca81138411f5f6db2d91c74ce12f5173205b513f6f13165518" exitCode=0 Jul 17 07:19:30.381318 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:30.380934 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-r24kq" Jul 17 07:19:30.381318 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:30.380949 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-r24kq" event={"ID":"97870d03-571c-4ea4-a322-7f082f436ea5","Type":"ContainerDied","Data":"20cf51c7b30d0eca81138411f5f6db2d91c74ce12f5173205b513f6f13165518"} Jul 17 07:19:30.381318 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:30.381028 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-r24kq" event={"ID":"97870d03-571c-4ea4-a322-7f082f436ea5","Type":"ContainerDied","Data":"9f6f8c4d47a0461abc6f25c7b21581d12f64eca53f4b9945d676ce9af8a04fc5"} Jul 17 07:19:30.381318 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:30.381051 2568 scope.go:117] "RemoveContainer" containerID="20cf51c7b30d0eca81138411f5f6db2d91c74ce12f5173205b513f6f13165518" Jul 17 07:19:30.390390 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:30.390360 2568 scope.go:117] "RemoveContainer" containerID="20cf51c7b30d0eca81138411f5f6db2d91c74ce12f5173205b513f6f13165518" Jul 17 07:19:30.390630 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:19:30.390610 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20cf51c7b30d0eca81138411f5f6db2d91c74ce12f5173205b513f6f13165518\": container with ID starting with 20cf51c7b30d0eca81138411f5f6db2d91c74ce12f5173205b513f6f13165518 not found: ID does not exist" containerID="20cf51c7b30d0eca81138411f5f6db2d91c74ce12f5173205b513f6f13165518" Jul 17 07:19:30.390675 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:30.390636 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20cf51c7b30d0eca81138411f5f6db2d91c74ce12f5173205b513f6f13165518"} err="failed to get container status \"20cf51c7b30d0eca81138411f5f6db2d91c74ce12f5173205b513f6f13165518\": rpc error: code = NotFound desc = could not find container \"20cf51c7b30d0eca81138411f5f6db2d91c74ce12f5173205b513f6f13165518\": container with ID starting with 20cf51c7b30d0eca81138411f5f6db2d91c74ce12f5173205b513f6f13165518 not found: ID does not exist" Jul 17 07:19:30.399025 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:30.398999 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-r24kq"] Jul 17 07:19:30.403085 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:30.403063 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-r24kq"] Jul 17 07:19:30.994525 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:30.994493 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-smfbz"] Jul 17 07:19:30.994875 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:30.994861 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1b06560-ac99-4177-8775-a4f1750ed64c" containerName="authorino" Jul 17 07:19:30.994943 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:30.994877 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1b06560-ac99-4177-8775-a4f1750ed64c" containerName="authorino" Jul 17 07:19:30.994943 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:30.994887 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97870d03-571c-4ea4-a322-7f082f436ea5" containerName="limitador" Jul 17 07:19:30.994943 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:30.994892 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="97870d03-571c-4ea4-a322-7f082f436ea5" containerName="limitador" Jul 17 07:19:30.995039 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:30.994969 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1b06560-ac99-4177-8775-a4f1750ed64c" containerName="authorino" Jul 17 07:19:30.995039 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:30.994980 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="97870d03-571c-4ea4-a322-7f082f436ea5" containerName="limitador" Jul 17 07:19:30.999448 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:30.999431 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-smfbz" Jul 17 07:19:31.002757 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:31.002732 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 17 07:19:31.002872 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:31.002735 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 17 07:19:31.003048 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:31.003031 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-z7j7z\"" Jul 17 07:19:31.003617 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:31.003603 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 17 07:19:31.014832 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:31.014809 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-smfbz"] Jul 17 07:19:31.107703 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:31.107663 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/b2a05aa6-6f0b-43f0-aa68-79996fd6de35-data\") pod \"postgres-69fb4594cf-smfbz\" (UID: \"b2a05aa6-6f0b-43f0-aa68-79996fd6de35\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-smfbz" Jul 17 07:19:31.107934 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:31.107726 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cvcw\" (UniqueName: \"kubernetes.io/projected/b2a05aa6-6f0b-43f0-aa68-79996fd6de35-kube-api-access-7cvcw\") pod \"postgres-69fb4594cf-smfbz\" (UID: \"b2a05aa6-6f0b-43f0-aa68-79996fd6de35\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-smfbz" Jul 17 07:19:31.208550 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:31.208517 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/b2a05aa6-6f0b-43f0-aa68-79996fd6de35-data\") pod \"postgres-69fb4594cf-smfbz\" (UID: \"b2a05aa6-6f0b-43f0-aa68-79996fd6de35\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-smfbz" Jul 17 07:19:31.208715 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:31.208562 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cvcw\" (UniqueName: \"kubernetes.io/projected/b2a05aa6-6f0b-43f0-aa68-79996fd6de35-kube-api-access-7cvcw\") pod \"postgres-69fb4594cf-smfbz\" (UID: \"b2a05aa6-6f0b-43f0-aa68-79996fd6de35\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-smfbz" Jul 17 07:19:31.208989 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:31.208968 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/b2a05aa6-6f0b-43f0-aa68-79996fd6de35-data\") pod \"postgres-69fb4594cf-smfbz\" (UID: \"b2a05aa6-6f0b-43f0-aa68-79996fd6de35\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-smfbz" Jul 17 07:19:31.218020 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:31.217993 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cvcw\" (UniqueName: \"kubernetes.io/projected/b2a05aa6-6f0b-43f0-aa68-79996fd6de35-kube-api-access-7cvcw\") pod \"postgres-69fb4594cf-smfbz\" (UID: \"b2a05aa6-6f0b-43f0-aa68-79996fd6de35\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-smfbz" Jul 17 07:19:31.310042 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:31.309957 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-smfbz" Jul 17 07:19:31.442774 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:31.442731 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-smfbz"] Jul 17 07:19:31.445708 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:19:31.445678 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2a05aa6_6f0b_43f0_aa68_79996fd6de35.slice/crio-a5d1a74bb9175afd78d211fc55d2de540c44b2c00469d73b90be16bd56d711ae WatchSource:0}: Error finding container a5d1a74bb9175afd78d211fc55d2de540c44b2c00469d73b90be16bd56d711ae: Status 404 returned error can't find the container with id a5d1a74bb9175afd78d211fc55d2de540c44b2c00469d73b90be16bd56d711ae Jul 17 07:19:32.000024 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:31.999992 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97870d03-571c-4ea4-a322-7f082f436ea5" path="/var/lib/kubelet/pods/97870d03-571c-4ea4-a322-7f082f436ea5/volumes" Jul 17 07:19:32.394936 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:32.394901 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-smfbz" event={"ID":"b2a05aa6-6f0b-43f0-aa68-79996fd6de35","Type":"ContainerStarted","Data":"a5d1a74bb9175afd78d211fc55d2de540c44b2c00469d73b90be16bd56d711ae"} Jul 17 07:19:47.411918 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:47.411890 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 17 07:19:48.482743 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:48.482701 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-smfbz" event={"ID":"b2a05aa6-6f0b-43f0-aa68-79996fd6de35","Type":"ContainerStarted","Data":"38bcd71e9a58e2adeb55a150d24fed8bb6a2128af4d0c4d9bd6ce7f389603a22"} Jul 17 07:19:48.483130 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:48.482771 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-smfbz" Jul 17 07:19:48.502070 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:48.502015 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-smfbz" podStartSLOduration=2.5399277319999998 podStartE2EDuration="18.50200015s" totalImagesPullingTime="15.962072418s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 07:19:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:19:48.499717245 +0000 UTC m=+492.984406219" watchObservedRunningTime="2026-07-17 07:19:48.50200015 +0000 UTC m=+492.986689124" Jul 17 07:19:54.517494 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:54.517466 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-smfbz" Jul 17 07:19:55.097764 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:55.097726 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-wsg7s"] Jul 17 07:19:55.105751 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:55.105727 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-wsg7s" Jul 17 07:19:55.106222 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:55.106193 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-wsg7s"] Jul 17 07:19:55.108040 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:55.108023 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-2kklx\"" Jul 17 07:19:55.219227 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:55.219196 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7tgs\" (UniqueName: \"kubernetes.io/projected/5a1acf9e-f1bf-4d59-a35a-b020bf87587a-kube-api-access-l7tgs\") pod \"authorino-5d4b67c446-wsg7s\" (UID: \"5a1acf9e-f1bf-4d59-a35a-b020bf87587a\") " pod="kuadrant-system/authorino-5d4b67c446-wsg7s" Jul 17 07:19:55.290554 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:55.290524 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-wsg7s"] Jul 17 07:19:55.290796 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:19:55.290778 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-l7tgs], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-wsg7s" podUID="5a1acf9e-f1bf-4d59-a35a-b020bf87587a" Jul 17 07:19:55.319496 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:55.319460 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-76877994-fggwj"] Jul 17 07:19:55.319799 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:55.319775 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7tgs\" (UniqueName: \"kubernetes.io/projected/5a1acf9e-f1bf-4d59-a35a-b020bf87587a-kube-api-access-l7tgs\") pod \"authorino-5d4b67c446-wsg7s\" (UID: \"5a1acf9e-f1bf-4d59-a35a-b020bf87587a\") " pod="kuadrant-system/authorino-5d4b67c446-wsg7s" Jul 17 07:19:55.323100 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:55.323075 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-fggwj" Jul 17 07:19:55.325595 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:55.325574 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 17 07:19:55.330991 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:55.330972 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76877994-fggwj"] Jul 17 07:19:55.333083 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:55.333060 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7tgs\" (UniqueName: \"kubernetes.io/projected/5a1acf9e-f1bf-4d59-a35a-b020bf87587a-kube-api-access-l7tgs\") pod \"authorino-5d4b67c446-wsg7s\" (UID: \"5a1acf9e-f1bf-4d59-a35a-b020bf87587a\") " pod="kuadrant-system/authorino-5d4b67c446-wsg7s" Jul 17 07:19:55.380953 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:55.380920 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-fggwj"] Jul 17 07:19:55.381174 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:19:55.381153 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-v9wx5 tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-76877994-fggwj" podUID="d8785bce-c186-41fd-8506-8dad33a0d2a2" Jul 17 07:19:55.410232 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:55.410207 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7b9959cbb5-nlj9t"] Jul 17 07:19:55.413746 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:55.413730 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b9959cbb5-nlj9t" Jul 17 07:19:55.420180 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:55.420157 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d8785bce-c186-41fd-8506-8dad33a0d2a2-tls-cert\") pod \"authorino-76877994-fggwj\" (UID: \"d8785bce-c186-41fd-8506-8dad33a0d2a2\") " pod="kuadrant-system/authorino-76877994-fggwj" Jul 17 07:19:55.420330 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:55.420231 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9wx5\" (UniqueName: \"kubernetes.io/projected/d8785bce-c186-41fd-8506-8dad33a0d2a2-kube-api-access-v9wx5\") pod \"authorino-76877994-fggwj\" (UID: \"d8785bce-c186-41fd-8506-8dad33a0d2a2\") " pod="kuadrant-system/authorino-76877994-fggwj" Jul 17 07:19:55.423602 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:55.423580 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7b9959cbb5-nlj9t"] Jul 17 07:19:55.519491 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:55.519458 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-fggwj" Jul 17 07:19:55.519909 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:55.519469 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-wsg7s" Jul 17 07:19:55.520692 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:55.520673 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rztgq\" (UniqueName: \"kubernetes.io/projected/a1fcc4cb-425d-45d6-9651-bea96fdbc004-kube-api-access-rztgq\") pod \"authorino-7b9959cbb5-nlj9t\" (UID: \"a1fcc4cb-425d-45d6-9651-bea96fdbc004\") " pod="kuadrant-system/authorino-7b9959cbb5-nlj9t" Jul 17 07:19:55.520784 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:55.520767 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d8785bce-c186-41fd-8506-8dad33a0d2a2-tls-cert\") pod \"authorino-76877994-fggwj\" (UID: \"d8785bce-c186-41fd-8506-8dad33a0d2a2\") " pod="kuadrant-system/authorino-76877994-fggwj" Jul 17 07:19:55.520869 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:55.520853 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9wx5\" (UniqueName: \"kubernetes.io/projected/d8785bce-c186-41fd-8506-8dad33a0d2a2-kube-api-access-v9wx5\") pod \"authorino-76877994-fggwj\" (UID: \"d8785bce-c186-41fd-8506-8dad33a0d2a2\") " pod="kuadrant-system/authorino-76877994-fggwj" Jul 17 07:19:55.520930 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:55.520892 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a1fcc4cb-425d-45d6-9651-bea96fdbc004-tls-cert\") pod \"authorino-7b9959cbb5-nlj9t\" (UID: \"a1fcc4cb-425d-45d6-9651-bea96fdbc004\") " pod="kuadrant-system/authorino-7b9959cbb5-nlj9t" Jul 17 07:19:55.523182 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:55.523155 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d8785bce-c186-41fd-8506-8dad33a0d2a2-tls-cert\") pod \"authorino-76877994-fggwj\" (UID: \"d8785bce-c186-41fd-8506-8dad33a0d2a2\") " pod="kuadrant-system/authorino-76877994-fggwj" Jul 17 07:19:55.526266 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:55.526249 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-fggwj" Jul 17 07:19:55.529228 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:55.529204 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9wx5\" (UniqueName: \"kubernetes.io/projected/d8785bce-c186-41fd-8506-8dad33a0d2a2-kube-api-access-v9wx5\") pod \"authorino-76877994-fggwj\" (UID: \"d8785bce-c186-41fd-8506-8dad33a0d2a2\") " pod="kuadrant-system/authorino-76877994-fggwj" Jul 17 07:19:55.543998 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:55.543980 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-wsg7s" Jul 17 07:19:55.621825 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:55.621798 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v9wx5\" (UniqueName: \"kubernetes.io/projected/d8785bce-c186-41fd-8506-8dad33a0d2a2-kube-api-access-v9wx5\") pod \"d8785bce-c186-41fd-8506-8dad33a0d2a2\" (UID: \"d8785bce-c186-41fd-8506-8dad33a0d2a2\") " Jul 17 07:19:55.621992 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:55.621896 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l7tgs\" (UniqueName: \"kubernetes.io/projected/5a1acf9e-f1bf-4d59-a35a-b020bf87587a-kube-api-access-l7tgs\") pod \"5a1acf9e-f1bf-4d59-a35a-b020bf87587a\" (UID: \"5a1acf9e-f1bf-4d59-a35a-b020bf87587a\") " Jul 17 07:19:55.621992 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:55.621926 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d8785bce-c186-41fd-8506-8dad33a0d2a2-tls-cert\") pod \"d8785bce-c186-41fd-8506-8dad33a0d2a2\" (UID: \"d8785bce-c186-41fd-8506-8dad33a0d2a2\") " Jul 17 07:19:55.622234 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:55.622066 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a1fcc4cb-425d-45d6-9651-bea96fdbc004-tls-cert\") pod \"authorino-7b9959cbb5-nlj9t\" (UID: \"a1fcc4cb-425d-45d6-9651-bea96fdbc004\") " pod="kuadrant-system/authorino-7b9959cbb5-nlj9t" Jul 17 07:19:55.622234 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:55.622130 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rztgq\" (UniqueName: \"kubernetes.io/projected/a1fcc4cb-425d-45d6-9651-bea96fdbc004-kube-api-access-rztgq\") pod \"authorino-7b9959cbb5-nlj9t\" (UID: \"a1fcc4cb-425d-45d6-9651-bea96fdbc004\") " pod="kuadrant-system/authorino-7b9959cbb5-nlj9t" Jul 17 07:19:55.624072 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:55.624038 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a1acf9e-f1bf-4d59-a35a-b020bf87587a-kube-api-access-l7tgs" (OuterVolumeSpecName: "kube-api-access-l7tgs") pod "5a1acf9e-f1bf-4d59-a35a-b020bf87587a" (UID: "5a1acf9e-f1bf-4d59-a35a-b020bf87587a"). InnerVolumeSpecName "kube-api-access-l7tgs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 07:19:55.624072 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:55.624068 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8785bce-c186-41fd-8506-8dad33a0d2a2-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "d8785bce-c186-41fd-8506-8dad33a0d2a2" (UID: "d8785bce-c186-41fd-8506-8dad33a0d2a2"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 07:19:55.624244 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:55.624078 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8785bce-c186-41fd-8506-8dad33a0d2a2-kube-api-access-v9wx5" (OuterVolumeSpecName: "kube-api-access-v9wx5") pod "d8785bce-c186-41fd-8506-8dad33a0d2a2" (UID: "d8785bce-c186-41fd-8506-8dad33a0d2a2"). InnerVolumeSpecName "kube-api-access-v9wx5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 07:19:55.624518 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:55.624500 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a1fcc4cb-425d-45d6-9651-bea96fdbc004-tls-cert\") pod \"authorino-7b9959cbb5-nlj9t\" (UID: \"a1fcc4cb-425d-45d6-9651-bea96fdbc004\") " pod="kuadrant-system/authorino-7b9959cbb5-nlj9t" Jul 17 07:19:55.630357 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:55.630338 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rztgq\" (UniqueName: \"kubernetes.io/projected/a1fcc4cb-425d-45d6-9651-bea96fdbc004-kube-api-access-rztgq\") pod \"authorino-7b9959cbb5-nlj9t\" (UID: \"a1fcc4cb-425d-45d6-9651-bea96fdbc004\") " pod="kuadrant-system/authorino-7b9959cbb5-nlj9t" Jul 17 07:19:55.723348 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:55.723276 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b9959cbb5-nlj9t" Jul 17 07:19:55.723552 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:55.723535 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l7tgs\" (UniqueName: \"kubernetes.io/projected/5a1acf9e-f1bf-4d59-a35a-b020bf87587a-kube-api-access-l7tgs\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:19:55.723594 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:55.723555 2568 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d8785bce-c186-41fd-8506-8dad33a0d2a2-tls-cert\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:19:55.723594 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:55.723565 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v9wx5\" (UniqueName: \"kubernetes.io/projected/d8785bce-c186-41fd-8506-8dad33a0d2a2-kube-api-access-v9wx5\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:19:55.852465 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:55.852438 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7b9959cbb5-nlj9t"] Jul 17 07:19:55.853765 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:19:55.853728 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda1fcc4cb_425d_45d6_9651_bea96fdbc004.slice/crio-7e016b2e22171b5acff8646cdbf4f3ab0b3f3f5d170191984e1418836d5e397f WatchSource:0}: Error finding container 7e016b2e22171b5acff8646cdbf4f3ab0b3f3f5d170191984e1418836d5e397f: Status 404 returned error can't find the container with id 7e016b2e22171b5acff8646cdbf4f3ab0b3f3f5d170191984e1418836d5e397f Jul 17 07:19:56.526478 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:56.526444 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b9959cbb5-nlj9t" event={"ID":"a1fcc4cb-425d-45d6-9651-bea96fdbc004","Type":"ContainerStarted","Data":"4cd0457b49508bab0364e3a1c9b1dd13e01b8a516ab8cd7a40a7def623aeed19"} Jul 17 07:19:56.526478 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:56.526484 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b9959cbb5-nlj9t" event={"ID":"a1fcc4cb-425d-45d6-9651-bea96fdbc004","Type":"ContainerStarted","Data":"7e016b2e22171b5acff8646cdbf4f3ab0b3f3f5d170191984e1418836d5e397f"} Jul 17 07:19:56.527005 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:56.526498 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-wsg7s" Jul 17 07:19:56.527005 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:56.526516 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-fggwj" Jul 17 07:19:56.555496 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:56.555449 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7b9959cbb5-nlj9t" podStartSLOduration=1.226451814 podStartE2EDuration="1.555433503s" totalImagesPullingTime="328.981689ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 07:19:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:19:56.554065658 +0000 UTC m=+501.038754697" watchObservedRunningTime="2026-07-17 07:19:56.555433503 +0000 UTC m=+501.040122477" Jul 17 07:19:56.580462 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:56.580423 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-wsg7s"] Jul 17 07:19:56.591516 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:56.591488 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-wsg7s"] Jul 17 07:19:56.618713 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:56.618686 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-fggwj"] Jul 17 07:19:56.622818 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:56.622796 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-76877994-fggwj"] Jul 17 07:19:58.000090 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:58.000056 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a1acf9e-f1bf-4d59-a35a-b020bf87587a" path="/var/lib/kubelet/pods/5a1acf9e-f1bf-4d59-a35a-b020bf87587a/volumes" Jul 17 07:19:58.000477 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:19:58.000323 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8785bce-c186-41fd-8506-8dad33a0d2a2" path="/var/lib/kubelet/pods/d8785bce-c186-41fd-8506-8dad33a0d2a2/volumes" Jul 17 07:20:01.623498 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:01.623463 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-579d999db4-5zmck"] Jul 17 07:20:01.627213 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:01.627189 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579d999db4-5zmck" Jul 17 07:20:01.629751 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:01.629725 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 17 07:20:01.629885 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:01.629729 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 17 07:20:01.630531 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:01.630512 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-x65gb\"" Jul 17 07:20:01.639815 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:01.639791 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-579d999db4-5zmck"] Jul 17 07:20:01.774379 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:01.774339 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvhbh\" (UniqueName: \"kubernetes.io/projected/aa87ee63-d35d-41f2-b998-68660d30e8e2-kube-api-access-wvhbh\") pod \"maas-controller-579d999db4-5zmck\" (UID: \"aa87ee63-d35d-41f2-b998-68660d30e8e2\") " pod="opendatahub/maas-controller-579d999db4-5zmck" Jul 17 07:20:01.774551 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:01.774389 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aa87ee63-d35d-41f2-b998-68660d30e8e2-webhook-cert\") pod \"maas-controller-579d999db4-5zmck\" (UID: \"aa87ee63-d35d-41f2-b998-68660d30e8e2\") " pod="opendatahub/maas-controller-579d999db4-5zmck" Jul 17 07:20:01.875068 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:01.874977 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvhbh\" (UniqueName: \"kubernetes.io/projected/aa87ee63-d35d-41f2-b998-68660d30e8e2-kube-api-access-wvhbh\") pod \"maas-controller-579d999db4-5zmck\" (UID: \"aa87ee63-d35d-41f2-b998-68660d30e8e2\") " pod="opendatahub/maas-controller-579d999db4-5zmck" Jul 17 07:20:01.875068 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:01.875025 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aa87ee63-d35d-41f2-b998-68660d30e8e2-webhook-cert\") pod \"maas-controller-579d999db4-5zmck\" (UID: \"aa87ee63-d35d-41f2-b998-68660d30e8e2\") " pod="opendatahub/maas-controller-579d999db4-5zmck" Jul 17 07:20:01.877568 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:01.877535 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aa87ee63-d35d-41f2-b998-68660d30e8e2-webhook-cert\") pod \"maas-controller-579d999db4-5zmck\" (UID: \"aa87ee63-d35d-41f2-b998-68660d30e8e2\") " pod="opendatahub/maas-controller-579d999db4-5zmck" Jul 17 07:20:01.883776 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:01.883749 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvhbh\" (UniqueName: \"kubernetes.io/projected/aa87ee63-d35d-41f2-b998-68660d30e8e2-kube-api-access-wvhbh\") pod \"maas-controller-579d999db4-5zmck\" (UID: \"aa87ee63-d35d-41f2-b998-68660d30e8e2\") " pod="opendatahub/maas-controller-579d999db4-5zmck" Jul 17 07:20:01.938205 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:01.938171 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579d999db4-5zmck" Jul 17 07:20:02.078461 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:02.078429 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-579d999db4-5zmck"] Jul 17 07:20:02.081777 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:20:02.081747 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa87ee63_d35d_41f2_b998_68660d30e8e2.slice/crio-47f1909366a58ff19d11beca6b3f7d95e2400d76b64fddeb156fe61cd73e5440 WatchSource:0}: Error finding container 47f1909366a58ff19d11beca6b3f7d95e2400d76b64fddeb156fe61cd73e5440: Status 404 returned error can't find the container with id 47f1909366a58ff19d11beca6b3f7d95e2400d76b64fddeb156fe61cd73e5440 Jul 17 07:20:02.556452 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:02.556415 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579d999db4-5zmck" event={"ID":"aa87ee63-d35d-41f2-b998-68660d30e8e2","Type":"ContainerStarted","Data":"47f1909366a58ff19d11beca6b3f7d95e2400d76b64fddeb156fe61cd73e5440"} Jul 17 07:20:05.574874 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:05.574833 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579d999db4-5zmck" event={"ID":"aa87ee63-d35d-41f2-b998-68660d30e8e2","Type":"ContainerStarted","Data":"d2ac1210aa611d82937f82ee8253d3499cde868b5ea706eb53e9cb4c0bf59955"} Jul 17 07:20:05.575428 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:05.574924 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-579d999db4-5zmck" Jul 17 07:20:05.593897 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:05.593847 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-579d999db4-5zmck" podStartSLOduration=1.7100618239999998 podStartE2EDuration="4.593834083s" totalImagesPullingTime="2.883772259s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 07:20:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:20:05.592774029 +0000 UTC m=+510.077463003" watchObservedRunningTime="2026-07-17 07:20:05.593834083 +0000 UTC m=+510.078523107" Jul 17 07:20:16.582198 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:16.582168 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-579d999db4-5zmck" Jul 17 07:20:39.419397 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:39.419315 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7fffcb47b4-j9l8r"] Jul 17 07:20:39.425745 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:39.425723 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7fffcb47b4-j9l8r" Jul 17 07:20:39.428557 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:39.428532 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 17 07:20:39.428694 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:39.428670 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-jwckj\"" Jul 17 07:20:39.436721 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:39.436696 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7fffcb47b4-j9l8r"] Jul 17 07:20:39.501438 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:39.501410 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrtkh\" (UniqueName: \"kubernetes.io/projected/1f61d02c-327d-4010-97ec-737f365bd17a-kube-api-access-xrtkh\") pod \"maas-api-7fffcb47b4-j9l8r\" (UID: \"1f61d02c-327d-4010-97ec-737f365bd17a\") " pod="odh-ai-gateway-infra/maas-api-7fffcb47b4-j9l8r" Jul 17 07:20:39.501601 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:39.501456 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f61d02c-327d-4010-97ec-737f365bd17a-openshift-service-ca\") pod \"maas-api-7fffcb47b4-j9l8r\" (UID: \"1f61d02c-327d-4010-97ec-737f365bd17a\") " pod="odh-ai-gateway-infra/maas-api-7fffcb47b4-j9l8r" Jul 17 07:20:39.501601 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:39.501527 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f61d02c-327d-4010-97ec-737f365bd17a-maas-api-tls\") pod \"maas-api-7fffcb47b4-j9l8r\" (UID: \"1f61d02c-327d-4010-97ec-737f365bd17a\") " pod="odh-ai-gateway-infra/maas-api-7fffcb47b4-j9l8r" Jul 17 07:20:39.602111 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:39.602069 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f61d02c-327d-4010-97ec-737f365bd17a-maas-api-tls\") pod \"maas-api-7fffcb47b4-j9l8r\" (UID: \"1f61d02c-327d-4010-97ec-737f365bd17a\") " pod="odh-ai-gateway-infra/maas-api-7fffcb47b4-j9l8r" Jul 17 07:20:39.602342 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:39.602155 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrtkh\" (UniqueName: \"kubernetes.io/projected/1f61d02c-327d-4010-97ec-737f365bd17a-kube-api-access-xrtkh\") pod \"maas-api-7fffcb47b4-j9l8r\" (UID: \"1f61d02c-327d-4010-97ec-737f365bd17a\") " pod="odh-ai-gateway-infra/maas-api-7fffcb47b4-j9l8r" Jul 17 07:20:39.602342 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:39.602210 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f61d02c-327d-4010-97ec-737f365bd17a-openshift-service-ca\") pod \"maas-api-7fffcb47b4-j9l8r\" (UID: \"1f61d02c-327d-4010-97ec-737f365bd17a\") " pod="odh-ai-gateway-infra/maas-api-7fffcb47b4-j9l8r" Jul 17 07:20:39.602848 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:39.602823 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f61d02c-327d-4010-97ec-737f365bd17a-openshift-service-ca\") pod \"maas-api-7fffcb47b4-j9l8r\" (UID: \"1f61d02c-327d-4010-97ec-737f365bd17a\") " pod="odh-ai-gateway-infra/maas-api-7fffcb47b4-j9l8r" Jul 17 07:20:39.604585 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:39.604564 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f61d02c-327d-4010-97ec-737f365bd17a-maas-api-tls\") pod \"maas-api-7fffcb47b4-j9l8r\" (UID: \"1f61d02c-327d-4010-97ec-737f365bd17a\") " pod="odh-ai-gateway-infra/maas-api-7fffcb47b4-j9l8r" Jul 17 07:20:39.611548 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:39.611516 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrtkh\" (UniqueName: \"kubernetes.io/projected/1f61d02c-327d-4010-97ec-737f365bd17a-kube-api-access-xrtkh\") pod \"maas-api-7fffcb47b4-j9l8r\" (UID: \"1f61d02c-327d-4010-97ec-737f365bd17a\") " pod="odh-ai-gateway-infra/maas-api-7fffcb47b4-j9l8r" Jul 17 07:20:39.738323 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:39.738203 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7fffcb47b4-j9l8r" Jul 17 07:20:39.876705 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:39.876682 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7fffcb47b4-j9l8r"] Jul 17 07:20:39.878118 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:20:39.878087 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f61d02c_327d_4010_97ec_737f365bd17a.slice/crio-6e5ba7cfd7e0f66d6331bcc5245fe4e9b3d9e19e9880ef5418f475e9e85aaa7e WatchSource:0}: Error finding container 6e5ba7cfd7e0f66d6331bcc5245fe4e9b3d9e19e9880ef5418f475e9e85aaa7e: Status 404 returned error can't find the container with id 6e5ba7cfd7e0f66d6331bcc5245fe4e9b3d9e19e9880ef5418f475e9e85aaa7e Jul 17 07:20:40.163195 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:40.163161 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-gk6w2"] Jul 17 07:20:40.167812 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:40.167796 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-gk6w2" Jul 17 07:20:40.170142 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:40.170121 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-5xnjd\"" Jul 17 07:20:40.170211 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:40.170156 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 17 07:20:40.177138 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:40.177087 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-gk6w2"] Jul 17 07:20:40.308600 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:40.308564 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7sns\" (UniqueName: \"kubernetes.io/projected/43af3320-dce1-439c-b915-2db00fe459ea-kube-api-access-h7sns\") pod \"payload-pre-processing-7677b774bb-gk6w2\" (UID: \"43af3320-dce1-439c-b915-2db00fe459ea\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-gk6w2" Jul 17 07:20:40.308795 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:40.308726 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43af3320-dce1-439c-b915-2db00fe459ea-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-gk6w2\" (UID: \"43af3320-dce1-439c-b915-2db00fe459ea\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-gk6w2" Jul 17 07:20:40.409971 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:40.409929 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43af3320-dce1-439c-b915-2db00fe459ea-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-gk6w2\" (UID: \"43af3320-dce1-439c-b915-2db00fe459ea\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-gk6w2" Jul 17 07:20:40.410152 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:40.409989 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7sns\" (UniqueName: \"kubernetes.io/projected/43af3320-dce1-439c-b915-2db00fe459ea-kube-api-access-h7sns\") pod \"payload-pre-processing-7677b774bb-gk6w2\" (UID: \"43af3320-dce1-439c-b915-2db00fe459ea\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-gk6w2" Jul 17 07:20:40.410683 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:40.410657 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43af3320-dce1-439c-b915-2db00fe459ea-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-gk6w2\" (UID: \"43af3320-dce1-439c-b915-2db00fe459ea\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-gk6w2" Jul 17 07:20:40.419611 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:40.419541 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7sns\" (UniqueName: \"kubernetes.io/projected/43af3320-dce1-439c-b915-2db00fe459ea-kube-api-access-h7sns\") pod \"payload-pre-processing-7677b774bb-gk6w2\" (UID: \"43af3320-dce1-439c-b915-2db00fe459ea\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-gk6w2" Jul 17 07:20:40.479806 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:40.479778 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-gk6w2" Jul 17 07:20:40.616593 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:40.616558 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-gk6w2"] Jul 17 07:20:40.618246 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:20:40.618205 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43af3320_dce1_439c_b915_2db00fe459ea.slice/crio-bbd65a60703a69add401c5edaaea63d6293a5c3c5bcaa266016413e31cec13e0 WatchSource:0}: Error finding container bbd65a60703a69add401c5edaaea63d6293a5c3c5bcaa266016413e31cec13e0: Status 404 returned error can't find the container with id bbd65a60703a69add401c5edaaea63d6293a5c3c5bcaa266016413e31cec13e0 Jul 17 07:20:40.764631 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:40.764549 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-gk6w2" event={"ID":"43af3320-dce1-439c-b915-2db00fe459ea","Type":"ContainerStarted","Data":"bbd65a60703a69add401c5edaaea63d6293a5c3c5bcaa266016413e31cec13e0"} Jul 17 07:20:40.765823 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:40.765797 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7fffcb47b4-j9l8r" event={"ID":"1f61d02c-327d-4010-97ec-737f365bd17a","Type":"ContainerStarted","Data":"6e5ba7cfd7e0f66d6331bcc5245fe4e9b3d9e19e9880ef5418f475e9e85aaa7e"} Jul 17 07:20:42.797652 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:42.797613 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-gk6w2" event={"ID":"43af3320-dce1-439c-b915-2db00fe459ea","Type":"ContainerStarted","Data":"84377ce4d4dfd9231345ad8f2698c2de26d5150af466fad9031dd5a6ce5cb124"} Jul 17 07:20:42.798066 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:42.797720 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-gk6w2" Jul 17 07:20:42.799330 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:42.799305 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7fffcb47b4-j9l8r" event={"ID":"1f61d02c-327d-4010-97ec-737f365bd17a","Type":"ContainerStarted","Data":"8b5ff1ccfe8ae3cf5a2c82c82ad74de7ab5cb9bac157e0235c3f8b37e8173098"} Jul 17 07:20:42.799442 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:42.799356 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7fffcb47b4-j9l8r" Jul 17 07:20:42.816490 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:42.816444 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7677b774bb-gk6w2" podStartSLOduration=0.72578783 podStartE2EDuration="2.816429718s" totalImagesPullingTime="2.090641888s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 07:20:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:20:42.814378599 +0000 UTC m=+547.299067573" watchObservedRunningTime="2026-07-17 07:20:42.816429718 +0000 UTC m=+547.301118691" Jul 17 07:20:42.831514 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:42.831474 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7fffcb47b4-j9l8r" podStartSLOduration=1.7110881359999999 podStartE2EDuration="3.831462452s" totalImagesPullingTime="2.120374316s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 07:20:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:20:42.830307667 +0000 UTC m=+547.314996641" watchObservedRunningTime="2026-07-17 07:20:42.831462452 +0000 UTC m=+547.316151426" Jul 17 07:20:46.313481 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:46.313431 2568 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-zd7fg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 17 07:20:46.314046 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:46.313497 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zd7fg" podUID="c7852377-14b6-4b77-98d3-bc5546fdebe5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 17 07:20:47.834052 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:47.834020 2568 generic.go:356] "Generic (PLEG): container finished" podID="c7852377-14b6-4b77-98d3-bc5546fdebe5" containerID="4c2e8a642f829d82025a66c8e3f7f19f51bb76fd10bfcae4eaba50e8005023d7" exitCode=0 Jul 17 07:20:47.834453 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:47.834062 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zd7fg" event={"ID":"c7852377-14b6-4b77-98d3-bc5546fdebe5","Type":"ContainerDied","Data":"4c2e8a642f829d82025a66c8e3f7f19f51bb76fd10bfcae4eaba50e8005023d7"} Jul 17 07:20:47.834504 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:47.834461 2568 scope.go:117] "RemoveContainer" containerID="4c2e8a642f829d82025a66c8e3f7f19f51bb76fd10bfcae4eaba50e8005023d7" Jul 17 07:20:47.835526 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:47.835495 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:20:47.835593 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:47.835580 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:20:47.835644 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:47.835629 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:20:48.816710 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:48.816672 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7fffcb47b4-j9l8r" Jul 17 07:20:48.842183 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:48.842136 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zd7fg" event={"ID":"c7852377-14b6-4b77-98d3-bc5546fdebe5","Type":"ContainerStarted","Data":"c5d6099497692e66a631a056289771677955ebe37feb75733d74cdbf3381fbbc"} Jul 17 07:20:49.814232 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:49.814192 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zd7fg" Jul 17 07:20:49.814483 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:49.814456 2568 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-zd7fg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 17 07:20:49.814574 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:49.814519 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zd7fg" podUID="c7852377-14b6-4b77-98d3-bc5546fdebe5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 17 07:20:50.813697 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:50.813658 2568 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-zd7fg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 17 07:20:50.814112 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:50.813729 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zd7fg" podUID="c7852377-14b6-4b77-98d3-bc5546fdebe5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 17 07:20:51.814679 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:51.814640 2568 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-zd7fg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 17 07:20:51.815087 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:51.814718 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zd7fg" podUID="c7852377-14b6-4b77-98d3-bc5546fdebe5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 17 07:20:52.813973 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:52.813937 2568 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-zd7fg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 17 07:20:52.814147 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:52.813994 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zd7fg" podUID="c7852377-14b6-4b77-98d3-bc5546fdebe5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 17 07:20:53.813365 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:53.813333 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-gk6w2" Jul 17 07:20:53.817184 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:53.817159 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zd7fg" Jul 17 07:20:53.817512 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:53.817484 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zd7fg" Jul 17 07:20:53.818200 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:20:53.818185 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-zd7fg" Jul 17 07:21:05.046801 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:05.046767 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hxlmg"] Jul 17 07:21:05.050383 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:05.050366 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hxlmg" Jul 17 07:21:05.053894 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:05.053872 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 17 07:21:05.054028 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:05.053919 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 17 07:21:05.054028 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:05.053977 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 17 07:21:05.054191 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:05.054179 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-z7wkz\"" Jul 17 07:21:05.061874 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:05.061845 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hxlmg"] Jul 17 07:21:05.245565 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:05.245522 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de34f89f-7f80-4cd0-8842-65634e430cbd-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-hxlmg\" (UID: \"de34f89f-7f80-4cd0-8842-65634e430cbd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hxlmg" Jul 17 07:21:05.245738 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:05.245571 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de34f89f-7f80-4cd0-8842-65634e430cbd-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-hxlmg\" (UID: \"de34f89f-7f80-4cd0-8842-65634e430cbd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hxlmg" Jul 17 07:21:05.245738 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:05.245606 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57z52\" (UniqueName: \"kubernetes.io/projected/de34f89f-7f80-4cd0-8842-65634e430cbd-kube-api-access-57z52\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-hxlmg\" (UID: \"de34f89f-7f80-4cd0-8842-65634e430cbd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hxlmg" Jul 17 07:21:05.245738 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:05.245669 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de34f89f-7f80-4cd0-8842-65634e430cbd-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-hxlmg\" (UID: \"de34f89f-7f80-4cd0-8842-65634e430cbd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hxlmg" Jul 17 07:21:05.245738 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:05.245717 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de34f89f-7f80-4cd0-8842-65634e430cbd-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-hxlmg\" (UID: \"de34f89f-7f80-4cd0-8842-65634e430cbd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hxlmg" Jul 17 07:21:05.245883 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:05.245748 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de34f89f-7f80-4cd0-8842-65634e430cbd-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-hxlmg\" (UID: \"de34f89f-7f80-4cd0-8842-65634e430cbd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hxlmg" Jul 17 07:21:05.346934 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:05.346844 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de34f89f-7f80-4cd0-8842-65634e430cbd-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-hxlmg\" (UID: \"de34f89f-7f80-4cd0-8842-65634e430cbd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hxlmg" Jul 17 07:21:05.346934 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:05.346887 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de34f89f-7f80-4cd0-8842-65634e430cbd-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-hxlmg\" (UID: \"de34f89f-7f80-4cd0-8842-65634e430cbd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hxlmg" Jul 17 07:21:05.346934 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:05.346913 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57z52\" (UniqueName: \"kubernetes.io/projected/de34f89f-7f80-4cd0-8842-65634e430cbd-kube-api-access-57z52\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-hxlmg\" (UID: \"de34f89f-7f80-4cd0-8842-65634e430cbd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hxlmg" Jul 17 07:21:05.346934 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:05.346935 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de34f89f-7f80-4cd0-8842-65634e430cbd-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-hxlmg\" (UID: \"de34f89f-7f80-4cd0-8842-65634e430cbd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hxlmg" Jul 17 07:21:05.347317 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:05.346959 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de34f89f-7f80-4cd0-8842-65634e430cbd-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-hxlmg\" (UID: \"de34f89f-7f80-4cd0-8842-65634e430cbd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hxlmg" Jul 17 07:21:05.347317 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:05.346976 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de34f89f-7f80-4cd0-8842-65634e430cbd-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-hxlmg\" (UID: \"de34f89f-7f80-4cd0-8842-65634e430cbd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hxlmg" Jul 17 07:21:05.347317 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:05.347275 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de34f89f-7f80-4cd0-8842-65634e430cbd-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-hxlmg\" (UID: \"de34f89f-7f80-4cd0-8842-65634e430cbd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hxlmg" Jul 17 07:21:05.347464 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:05.347367 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de34f89f-7f80-4cd0-8842-65634e430cbd-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-hxlmg\" (UID: \"de34f89f-7f80-4cd0-8842-65634e430cbd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hxlmg" Jul 17 07:21:05.347553 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:05.347534 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de34f89f-7f80-4cd0-8842-65634e430cbd-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-hxlmg\" (UID: \"de34f89f-7f80-4cd0-8842-65634e430cbd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hxlmg" Jul 17 07:21:05.349416 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:05.349383 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de34f89f-7f80-4cd0-8842-65634e430cbd-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-hxlmg\" (UID: \"de34f89f-7f80-4cd0-8842-65634e430cbd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hxlmg" Jul 17 07:21:05.349416 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:05.349407 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de34f89f-7f80-4cd0-8842-65634e430cbd-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-hxlmg\" (UID: \"de34f89f-7f80-4cd0-8842-65634e430cbd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hxlmg" Jul 17 07:21:05.355423 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:05.355399 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-57z52\" (UniqueName: \"kubernetes.io/projected/de34f89f-7f80-4cd0-8842-65634e430cbd-kube-api-access-57z52\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-hxlmg\" (UID: \"de34f89f-7f80-4cd0-8842-65634e430cbd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hxlmg" Jul 17 07:21:05.369442 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:05.369422 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hxlmg" Jul 17 07:21:05.520938 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:05.520883 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hxlmg"] Jul 17 07:21:05.521907 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:21:05.521874 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde34f89f_7f80_4cd0_8842_65634e430cbd.slice/crio-eee94afce954cfaa0a9145d5e3c63fd6224d670f6da7e526e821ddc0659d5269 WatchSource:0}: Error finding container eee94afce954cfaa0a9145d5e3c63fd6224d670f6da7e526e821ddc0659d5269: Status 404 returned error can't find the container with id eee94afce954cfaa0a9145d5e3c63fd6224d670f6da7e526e821ddc0659d5269 Jul 17 07:21:05.946541 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:05.946497 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hxlmg" event={"ID":"de34f89f-7f80-4cd0-8842-65634e430cbd","Type":"ContainerStarted","Data":"eee94afce954cfaa0a9145d5e3c63fd6224d670f6da7e526e821ddc0659d5269"} Jul 17 07:21:06.953507 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:06.953473 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hxlmg" event={"ID":"de34f89f-7f80-4cd0-8842-65634e430cbd","Type":"ContainerStarted","Data":"8efa923725f725f14d5a86fde84b7120459943a5fd250c8cd277dcac486742e9"} Jul 17 07:21:06.972982 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:06.972932 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hxlmg" podStartSLOduration=0.746973386 podStartE2EDuration="1.972916976s" totalImagesPullingTime="1.22594359s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 07:21:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:21:06.971363532 +0000 UTC m=+571.456052505" watchObservedRunningTime="2026-07-17 07:21:06.972916976 +0000 UTC m=+571.457605950" Jul 17 07:21:15.370248 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:15.370210 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hxlmg" Jul 17 07:21:15.370789 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:15.370273 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hxlmg" Jul 17 07:21:15.384794 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:15.384764 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hxlmg" Jul 17 07:21:16.013295 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:16.013263 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hxlmg" Jul 17 07:21:35.974771 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:35.974736 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-xr6pg_2ce00a54-15e9-42b7-b3a8-9e58d1101558/cluster-monitoring-operator/0.log" Jul 17 07:21:35.979251 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:35.979224 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-xr6pg_2ce00a54-15e9-42b7-b3a8-9e58d1101558/cluster-monitoring-operator/0.log" Jul 17 07:21:35.979672 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:35.979653 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-xpxv5_27538d1e-992b-4651-a2d2-7a7ecf442806/console-operator/1.log" Jul 17 07:21:35.982895 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:35.982877 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cpwz4_1c44a3ae-4c83-4203-8b0b-116d6618accf/ovn-acl-logging/0.log" Jul 17 07:21:35.983970 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:35.983954 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-xpxv5_27538d1e-992b-4651-a2d2-7a7ecf442806/console-operator/1.log" Jul 17 07:21:35.990129 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:35.990113 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cpwz4_1c44a3ae-4c83-4203-8b0b-116d6618accf/ovn-acl-logging/0.log" Jul 17 07:21:44.140676 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:44.140627 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-bc6gv"] Jul 17 07:21:44.147431 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:44.147403 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bc6gv" Jul 17 07:21:44.150382 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:44.150360 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 17 07:21:44.156196 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:44.156173 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-bc6gv"] Jul 17 07:21:44.292724 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:44.292689 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b8af6fd-3d6c-48d6-9dde-bac6ce028c56-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bc6gv\" (UID: \"6b8af6fd-3d6c-48d6-9dde-bac6ce028c56\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bc6gv" Jul 17 07:21:44.292926 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:44.292771 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b8af6fd-3d6c-48d6-9dde-bac6ce028c56-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bc6gv\" (UID: \"6b8af6fd-3d6c-48d6-9dde-bac6ce028c56\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bc6gv" Jul 17 07:21:44.292926 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:44.292801 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9l8jm\" (UniqueName: \"kubernetes.io/projected/6b8af6fd-3d6c-48d6-9dde-bac6ce028c56-kube-api-access-9l8jm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bc6gv\" (UID: \"6b8af6fd-3d6c-48d6-9dde-bac6ce028c56\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bc6gv" Jul 17 07:21:44.292926 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:44.292849 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b8af6fd-3d6c-48d6-9dde-bac6ce028c56-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bc6gv\" (UID: \"6b8af6fd-3d6c-48d6-9dde-bac6ce028c56\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bc6gv" Jul 17 07:21:44.292926 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:44.292896 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b8af6fd-3d6c-48d6-9dde-bac6ce028c56-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bc6gv\" (UID: \"6b8af6fd-3d6c-48d6-9dde-bac6ce028c56\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bc6gv" Jul 17 07:21:44.293142 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:44.292947 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b8af6fd-3d6c-48d6-9dde-bac6ce028c56-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bc6gv\" (UID: \"6b8af6fd-3d6c-48d6-9dde-bac6ce028c56\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bc6gv" Jul 17 07:21:44.394456 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:44.394372 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b8af6fd-3d6c-48d6-9dde-bac6ce028c56-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bc6gv\" (UID: \"6b8af6fd-3d6c-48d6-9dde-bac6ce028c56\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bc6gv" Jul 17 07:21:44.394456 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:44.394434 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b8af6fd-3d6c-48d6-9dde-bac6ce028c56-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bc6gv\" (UID: \"6b8af6fd-3d6c-48d6-9dde-bac6ce028c56\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bc6gv" Jul 17 07:21:44.394686 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:44.394460 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9l8jm\" (UniqueName: \"kubernetes.io/projected/6b8af6fd-3d6c-48d6-9dde-bac6ce028c56-kube-api-access-9l8jm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bc6gv\" (UID: \"6b8af6fd-3d6c-48d6-9dde-bac6ce028c56\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bc6gv" Jul 17 07:21:44.394686 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:44.394498 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b8af6fd-3d6c-48d6-9dde-bac6ce028c56-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bc6gv\" (UID: \"6b8af6fd-3d6c-48d6-9dde-bac6ce028c56\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bc6gv" Jul 17 07:21:44.394686 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:44.394529 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b8af6fd-3d6c-48d6-9dde-bac6ce028c56-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bc6gv\" (UID: \"6b8af6fd-3d6c-48d6-9dde-bac6ce028c56\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bc6gv" Jul 17 07:21:44.394686 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:44.394567 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b8af6fd-3d6c-48d6-9dde-bac6ce028c56-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bc6gv\" (UID: \"6b8af6fd-3d6c-48d6-9dde-bac6ce028c56\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bc6gv" Jul 17 07:21:44.394932 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:44.394912 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b8af6fd-3d6c-48d6-9dde-bac6ce028c56-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bc6gv\" (UID: \"6b8af6fd-3d6c-48d6-9dde-bac6ce028c56\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bc6gv" Jul 17 07:21:44.394972 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:44.394936 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b8af6fd-3d6c-48d6-9dde-bac6ce028c56-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bc6gv\" (UID: \"6b8af6fd-3d6c-48d6-9dde-bac6ce028c56\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bc6gv" Jul 17 07:21:44.395014 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:44.394998 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b8af6fd-3d6c-48d6-9dde-bac6ce028c56-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bc6gv\" (UID: \"6b8af6fd-3d6c-48d6-9dde-bac6ce028c56\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bc6gv" Jul 17 07:21:44.396708 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:44.396681 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b8af6fd-3d6c-48d6-9dde-bac6ce028c56-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bc6gv\" (UID: \"6b8af6fd-3d6c-48d6-9dde-bac6ce028c56\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bc6gv" Jul 17 07:21:44.396946 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:44.396931 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b8af6fd-3d6c-48d6-9dde-bac6ce028c56-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bc6gv\" (UID: \"6b8af6fd-3d6c-48d6-9dde-bac6ce028c56\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bc6gv" Jul 17 07:21:44.402508 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:44.402482 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9l8jm\" (UniqueName: \"kubernetes.io/projected/6b8af6fd-3d6c-48d6-9dde-bac6ce028c56-kube-api-access-9l8jm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bc6gv\" (UID: \"6b8af6fd-3d6c-48d6-9dde-bac6ce028c56\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bc6gv" Jul 17 07:21:44.459969 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:44.459948 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bc6gv" Jul 17 07:21:44.601882 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:44.601851 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-bc6gv"] Jul 17 07:21:44.603031 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:21:44.603002 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b8af6fd_3d6c_48d6_9dde_bac6ce028c56.slice/crio-f72631899ea8a04c9000d53e4fa7623934ea9f2e0aa4b973934240cdd7335956 WatchSource:0}: Error finding container f72631899ea8a04c9000d53e4fa7623934ea9f2e0aa4b973934240cdd7335956: Status 404 returned error can't find the container with id f72631899ea8a04c9000d53e4fa7623934ea9f2e0aa4b973934240cdd7335956 Jul 17 07:21:44.604806 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:44.604782 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 17 07:21:45.153840 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:45.153805 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bc6gv" event={"ID":"6b8af6fd-3d6c-48d6-9dde-bac6ce028c56","Type":"ContainerStarted","Data":"f763049465ef7738191fad1f4ebe724298623360f6a9175167bc1a0d6ba6ce93"} Jul 17 07:21:45.154234 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:45.153856 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bc6gv" event={"ID":"6b8af6fd-3d6c-48d6-9dde-bac6ce028c56","Type":"ContainerStarted","Data":"f72631899ea8a04c9000d53e4fa7623934ea9f2e0aa4b973934240cdd7335956"} Jul 17 07:21:45.176604 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:45.176556 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bc6gv" podStartSLOduration=0.979414429 podStartE2EDuration="1.176540837s" totalImagesPullingTime="197.126408ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 07:21:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:21:45.175326435 +0000 UTC m=+609.660015412" watchObservedRunningTime="2026-07-17 07:21:45.176540837 +0000 UTC m=+609.661229873" Jul 17 07:21:54.460893 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:54.460856 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bc6gv" Jul 17 07:21:54.461495 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:54.460930 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bc6gv" Jul 17 07:21:54.473559 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:54.473537 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bc6gv" Jul 17 07:21:55.220441 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:21:55.220408 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bc6gv" Jul 17 07:22:38.963250 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:22:38.963217 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7b9959cbb5-nlj9t"] Jul 17 07:22:38.963814 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:22:38.963399 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7b9959cbb5-nlj9t" podUID="a1fcc4cb-425d-45d6-9651-bea96fdbc004" containerName="authorino" containerID="cri-o://4cd0457b49508bab0364e3a1c9b1dd13e01b8a516ab8cd7a40a7def623aeed19" gracePeriod=30 Jul 17 07:22:39.227186 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:22:39.227129 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b9959cbb5-nlj9t" Jul 17 07:22:39.377880 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:22:39.377838 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a1fcc4cb-425d-45d6-9651-bea96fdbc004-tls-cert\") pod \"a1fcc4cb-425d-45d6-9651-bea96fdbc004\" (UID: \"a1fcc4cb-425d-45d6-9651-bea96fdbc004\") " Jul 17 07:22:39.378110 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:22:39.377898 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rztgq\" (UniqueName: \"kubernetes.io/projected/a1fcc4cb-425d-45d6-9651-bea96fdbc004-kube-api-access-rztgq\") pod \"a1fcc4cb-425d-45d6-9651-bea96fdbc004\" (UID: \"a1fcc4cb-425d-45d6-9651-bea96fdbc004\") " Jul 17 07:22:39.379880 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:22:39.379828 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1fcc4cb-425d-45d6-9651-bea96fdbc004-kube-api-access-rztgq" (OuterVolumeSpecName: "kube-api-access-rztgq") pod "a1fcc4cb-425d-45d6-9651-bea96fdbc004" (UID: "a1fcc4cb-425d-45d6-9651-bea96fdbc004"). InnerVolumeSpecName "kube-api-access-rztgq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 07:22:39.388651 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:22:39.388625 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a1fcc4cb-425d-45d6-9651-bea96fdbc004-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "a1fcc4cb-425d-45d6-9651-bea96fdbc004" (UID: "a1fcc4cb-425d-45d6-9651-bea96fdbc004"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 07:22:39.450375 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:22:39.450337 2568 generic.go:356] "Generic (PLEG): container finished" podID="a1fcc4cb-425d-45d6-9651-bea96fdbc004" containerID="4cd0457b49508bab0364e3a1c9b1dd13e01b8a516ab8cd7a40a7def623aeed19" exitCode=0 Jul 17 07:22:39.450506 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:22:39.450387 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b9959cbb5-nlj9t" Jul 17 07:22:39.450506 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:22:39.450420 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b9959cbb5-nlj9t" event={"ID":"a1fcc4cb-425d-45d6-9651-bea96fdbc004","Type":"ContainerDied","Data":"4cd0457b49508bab0364e3a1c9b1dd13e01b8a516ab8cd7a40a7def623aeed19"} Jul 17 07:22:39.450506 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:22:39.450468 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b9959cbb5-nlj9t" event={"ID":"a1fcc4cb-425d-45d6-9651-bea96fdbc004","Type":"ContainerDied","Data":"7e016b2e22171b5acff8646cdbf4f3ab0b3f3f5d170191984e1418836d5e397f"} Jul 17 07:22:39.450506 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:22:39.450486 2568 scope.go:117] "RemoveContainer" containerID="4cd0457b49508bab0364e3a1c9b1dd13e01b8a516ab8cd7a40a7def623aeed19" Jul 17 07:22:39.460056 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:22:39.460040 2568 scope.go:117] "RemoveContainer" containerID="4cd0457b49508bab0364e3a1c9b1dd13e01b8a516ab8cd7a40a7def623aeed19" Jul 17 07:22:39.460352 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:22:39.460332 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4cd0457b49508bab0364e3a1c9b1dd13e01b8a516ab8cd7a40a7def623aeed19\": container with ID starting with 4cd0457b49508bab0364e3a1c9b1dd13e01b8a516ab8cd7a40a7def623aeed19 not found: ID does not exist" containerID="4cd0457b49508bab0364e3a1c9b1dd13e01b8a516ab8cd7a40a7def623aeed19" Jul 17 07:22:39.460408 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:22:39.460358 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4cd0457b49508bab0364e3a1c9b1dd13e01b8a516ab8cd7a40a7def623aeed19"} err="failed to get container status \"4cd0457b49508bab0364e3a1c9b1dd13e01b8a516ab8cd7a40a7def623aeed19\": rpc error: code = NotFound desc = could not find container \"4cd0457b49508bab0364e3a1c9b1dd13e01b8a516ab8cd7a40a7def623aeed19\": container with ID starting with 4cd0457b49508bab0364e3a1c9b1dd13e01b8a516ab8cd7a40a7def623aeed19 not found: ID does not exist" Jul 17 07:22:39.473647 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:22:39.473626 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7b9959cbb5-nlj9t"] Jul 17 07:22:39.477573 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:22:39.477524 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7b9959cbb5-nlj9t"] Jul 17 07:22:39.478607 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:22:39.478589 2568 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a1fcc4cb-425d-45d6-9651-bea96fdbc004-tls-cert\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:22:39.478660 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:22:39.478611 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rztgq\" (UniqueName: \"kubernetes.io/projected/a1fcc4cb-425d-45d6-9651-bea96fdbc004-kube-api-access-rztgq\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:22:40.000884 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:22:40.000842 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1fcc4cb-425d-45d6-9651-bea96fdbc004" path="/var/lib/kubelet/pods/a1fcc4cb-425d-45d6-9651-bea96fdbc004/volumes" Jul 17 07:23:24.775988 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:23:24.775955 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-579d999db4-5zmck"] Jul 17 07:23:24.776658 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:23:24.776146 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-579d999db4-5zmck" podUID="aa87ee63-d35d-41f2-b998-68660d30e8e2" containerName="manager" containerID="cri-o://d2ac1210aa611d82937f82ee8253d3499cde868b5ea706eb53e9cb4c0bf59955" gracePeriod=30 Jul 17 07:23:25.030772 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:23:25.030718 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579d999db4-5zmck" Jul 17 07:23:25.092413 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:23:25.092385 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aa87ee63-d35d-41f2-b998-68660d30e8e2-webhook-cert\") pod \"aa87ee63-d35d-41f2-b998-68660d30e8e2\" (UID: \"aa87ee63-d35d-41f2-b998-68660d30e8e2\") " Jul 17 07:23:25.094571 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:23:25.094547 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa87ee63-d35d-41f2-b998-68660d30e8e2-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "aa87ee63-d35d-41f2-b998-68660d30e8e2" (UID: "aa87ee63-d35d-41f2-b998-68660d30e8e2"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 07:23:25.193322 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:23:25.193272 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wvhbh\" (UniqueName: \"kubernetes.io/projected/aa87ee63-d35d-41f2-b998-68660d30e8e2-kube-api-access-wvhbh\") pod \"aa87ee63-d35d-41f2-b998-68660d30e8e2\" (UID: \"aa87ee63-d35d-41f2-b998-68660d30e8e2\") " Jul 17 07:23:25.193567 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:23:25.193551 2568 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aa87ee63-d35d-41f2-b998-68660d30e8e2-webhook-cert\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:23:25.195195 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:23:25.195170 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa87ee63-d35d-41f2-b998-68660d30e8e2-kube-api-access-wvhbh" (OuterVolumeSpecName: "kube-api-access-wvhbh") pod "aa87ee63-d35d-41f2-b998-68660d30e8e2" (UID: "aa87ee63-d35d-41f2-b998-68660d30e8e2"). InnerVolumeSpecName "kube-api-access-wvhbh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 07:23:25.294537 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:23:25.294463 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wvhbh\" (UniqueName: \"kubernetes.io/projected/aa87ee63-d35d-41f2-b998-68660d30e8e2-kube-api-access-wvhbh\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:23:25.710591 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:23:25.710553 2568 generic.go:356] "Generic (PLEG): container finished" podID="aa87ee63-d35d-41f2-b998-68660d30e8e2" containerID="d2ac1210aa611d82937f82ee8253d3499cde868b5ea706eb53e9cb4c0bf59955" exitCode=0 Jul 17 07:23:25.710770 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:23:25.710626 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579d999db4-5zmck" event={"ID":"aa87ee63-d35d-41f2-b998-68660d30e8e2","Type":"ContainerDied","Data":"d2ac1210aa611d82937f82ee8253d3499cde868b5ea706eb53e9cb4c0bf59955"} Jul 17 07:23:25.710770 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:23:25.710650 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579d999db4-5zmck" Jul 17 07:23:25.710770 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:23:25.710663 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579d999db4-5zmck" event={"ID":"aa87ee63-d35d-41f2-b998-68660d30e8e2","Type":"ContainerDied","Data":"47f1909366a58ff19d11beca6b3f7d95e2400d76b64fddeb156fe61cd73e5440"} Jul 17 07:23:25.710770 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:23:25.710687 2568 scope.go:117] "RemoveContainer" containerID="d2ac1210aa611d82937f82ee8253d3499cde868b5ea706eb53e9cb4c0bf59955" Jul 17 07:23:25.719788 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:23:25.719771 2568 scope.go:117] "RemoveContainer" containerID="d2ac1210aa611d82937f82ee8253d3499cde868b5ea706eb53e9cb4c0bf59955" Jul 17 07:23:25.720027 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:23:25.720009 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2ac1210aa611d82937f82ee8253d3499cde868b5ea706eb53e9cb4c0bf59955\": container with ID starting with d2ac1210aa611d82937f82ee8253d3499cde868b5ea706eb53e9cb4c0bf59955 not found: ID does not exist" containerID="d2ac1210aa611d82937f82ee8253d3499cde868b5ea706eb53e9cb4c0bf59955" Jul 17 07:23:25.720066 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:23:25.720035 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2ac1210aa611d82937f82ee8253d3499cde868b5ea706eb53e9cb4c0bf59955"} err="failed to get container status \"d2ac1210aa611d82937f82ee8253d3499cde868b5ea706eb53e9cb4c0bf59955\": rpc error: code = NotFound desc = could not find container \"d2ac1210aa611d82937f82ee8253d3499cde868b5ea706eb53e9cb4c0bf59955\": container with ID starting with d2ac1210aa611d82937f82ee8253d3499cde868b5ea706eb53e9cb4c0bf59955 not found: ID does not exist" Jul 17 07:23:25.733374 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:23:25.733352 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-579d999db4-5zmck"] Jul 17 07:23:25.739227 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:23:25.739207 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-579d999db4-5zmck"] Jul 17 07:23:26.000169 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:23:26.000093 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa87ee63-d35d-41f2-b998-68660d30e8e2" path="/var/lib/kubelet/pods/aa87ee63-d35d-41f2-b998-68660d30e8e2/volumes" Jul 17 07:24:26.568471 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:26.568437 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-579d999db4-kbs9l"] Jul 17 07:24:26.568923 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:26.568848 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1fcc4cb-425d-45d6-9651-bea96fdbc004" containerName="authorino" Jul 17 07:24:26.568923 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:26.568864 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1fcc4cb-425d-45d6-9651-bea96fdbc004" containerName="authorino" Jul 17 07:24:26.568923 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:26.568884 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa87ee63-d35d-41f2-b998-68660d30e8e2" containerName="manager" Jul 17 07:24:26.568923 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:26.568889 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa87ee63-d35d-41f2-b998-68660d30e8e2" containerName="manager" Jul 17 07:24:26.569057 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:26.568947 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1fcc4cb-425d-45d6-9651-bea96fdbc004" containerName="authorino" Jul 17 07:24:26.569057 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:26.568954 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa87ee63-d35d-41f2-b998-68660d30e8e2" containerName="manager" Jul 17 07:24:26.572065 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:26.572049 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579d999db4-kbs9l" Jul 17 07:24:26.575481 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:26.575458 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 17 07:24:26.575620 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:26.575458 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-x65gb\"" Jul 17 07:24:26.575620 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:26.575530 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 17 07:24:26.589753 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:26.589718 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-579d999db4-kbs9l"] Jul 17 07:24:26.616337 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:26.616306 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/30316be2-7e96-4ccb-82b2-16332b3b848d-webhook-cert\") pod \"maas-controller-579d999db4-kbs9l\" (UID: \"30316be2-7e96-4ccb-82b2-16332b3b848d\") " pod="opendatahub/maas-controller-579d999db4-kbs9l" Jul 17 07:24:26.616489 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:26.616346 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zz4w\" (UniqueName: \"kubernetes.io/projected/30316be2-7e96-4ccb-82b2-16332b3b848d-kube-api-access-9zz4w\") pod \"maas-controller-579d999db4-kbs9l\" (UID: \"30316be2-7e96-4ccb-82b2-16332b3b848d\") " pod="opendatahub/maas-controller-579d999db4-kbs9l" Jul 17 07:24:26.717368 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:26.717335 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/30316be2-7e96-4ccb-82b2-16332b3b848d-webhook-cert\") pod \"maas-controller-579d999db4-kbs9l\" (UID: \"30316be2-7e96-4ccb-82b2-16332b3b848d\") " pod="opendatahub/maas-controller-579d999db4-kbs9l" Jul 17 07:24:26.717545 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:26.717374 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zz4w\" (UniqueName: \"kubernetes.io/projected/30316be2-7e96-4ccb-82b2-16332b3b848d-kube-api-access-9zz4w\") pod \"maas-controller-579d999db4-kbs9l\" (UID: \"30316be2-7e96-4ccb-82b2-16332b3b848d\") " pod="opendatahub/maas-controller-579d999db4-kbs9l" Jul 17 07:24:26.719825 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:26.719797 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/30316be2-7e96-4ccb-82b2-16332b3b848d-webhook-cert\") pod \"maas-controller-579d999db4-kbs9l\" (UID: \"30316be2-7e96-4ccb-82b2-16332b3b848d\") " pod="opendatahub/maas-controller-579d999db4-kbs9l" Jul 17 07:24:26.726056 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:26.726033 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zz4w\" (UniqueName: \"kubernetes.io/projected/30316be2-7e96-4ccb-82b2-16332b3b848d-kube-api-access-9zz4w\") pod \"maas-controller-579d999db4-kbs9l\" (UID: \"30316be2-7e96-4ccb-82b2-16332b3b848d\") " pod="opendatahub/maas-controller-579d999db4-kbs9l" Jul 17 07:24:26.884254 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:26.884216 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579d999db4-kbs9l" Jul 17 07:24:27.018635 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:27.018609 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-579d999db4-kbs9l"] Jul 17 07:24:27.021146 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:24:27.021118 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30316be2_7e96_4ccb_82b2_16332b3b848d.slice/crio-19b649bb2f49357a576f135dfc37efab2c57a53fef03bb4d2da08ae6b3af4abb WatchSource:0}: Error finding container 19b649bb2f49357a576f135dfc37efab2c57a53fef03bb4d2da08ae6b3af4abb: Status 404 returned error can't find the container with id 19b649bb2f49357a576f135dfc37efab2c57a53fef03bb4d2da08ae6b3af4abb Jul 17 07:24:27.047976 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:27.047941 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579d999db4-kbs9l" event={"ID":"30316be2-7e96-4ccb-82b2-16332b3b848d","Type":"ContainerStarted","Data":"19b649bb2f49357a576f135dfc37efab2c57a53fef03bb4d2da08ae6b3af4abb"} Jul 17 07:24:28.055594 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:28.055554 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579d999db4-kbs9l" event={"ID":"30316be2-7e96-4ccb-82b2-16332b3b848d","Type":"ContainerStarted","Data":"86834a171ea51b8fde21eb54e28ee5bca60eda1ef84e0d51c18c87480539738f"} Jul 17 07:24:28.056009 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:28.055625 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-579d999db4-kbs9l" Jul 17 07:24:28.076805 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:28.076752 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-579d999db4-kbs9l" podStartSLOduration=1.673683048 podStartE2EDuration="2.076735827s" totalImagesPullingTime="403.052779ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 07:24:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:24:28.07596402 +0000 UTC m=+772.560652996" watchObservedRunningTime="2026-07-17 07:24:28.076735827 +0000 UTC m=+772.561424800" Jul 17 07:24:39.064598 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:39.064565 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-579d999db4-kbs9l" Jul 17 07:24:43.706672 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:43.706625 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-579d999db4-kbs9l"] Jul 17 07:24:43.707154 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:43.706830 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-579d999db4-kbs9l" podUID="30316be2-7e96-4ccb-82b2-16332b3b848d" containerName="manager" containerID="cri-o://86834a171ea51b8fde21eb54e28ee5bca60eda1ef84e0d51c18c87480539738f" gracePeriod=30 Jul 17 07:24:43.958821 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:43.958764 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579d999db4-kbs9l" Jul 17 07:24:44.068382 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:44.068354 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/30316be2-7e96-4ccb-82b2-16332b3b848d-webhook-cert\") pod \"30316be2-7e96-4ccb-82b2-16332b3b848d\" (UID: \"30316be2-7e96-4ccb-82b2-16332b3b848d\") " Jul 17 07:24:44.068537 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:44.068438 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9zz4w\" (UniqueName: \"kubernetes.io/projected/30316be2-7e96-4ccb-82b2-16332b3b848d-kube-api-access-9zz4w\") pod \"30316be2-7e96-4ccb-82b2-16332b3b848d\" (UID: \"30316be2-7e96-4ccb-82b2-16332b3b848d\") " Jul 17 07:24:44.070295 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:44.070258 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30316be2-7e96-4ccb-82b2-16332b3b848d-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "30316be2-7e96-4ccb-82b2-16332b3b848d" (UID: "30316be2-7e96-4ccb-82b2-16332b3b848d"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 07:24:44.070453 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:44.070430 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30316be2-7e96-4ccb-82b2-16332b3b848d-kube-api-access-9zz4w" (OuterVolumeSpecName: "kube-api-access-9zz4w") pod "30316be2-7e96-4ccb-82b2-16332b3b848d" (UID: "30316be2-7e96-4ccb-82b2-16332b3b848d"). InnerVolumeSpecName "kube-api-access-9zz4w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 07:24:44.144517 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:44.144486 2568 generic.go:356] "Generic (PLEG): container finished" podID="30316be2-7e96-4ccb-82b2-16332b3b848d" containerID="86834a171ea51b8fde21eb54e28ee5bca60eda1ef84e0d51c18c87480539738f" exitCode=0 Jul 17 07:24:44.144653 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:44.144533 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579d999db4-kbs9l" Jul 17 07:24:44.144653 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:44.144574 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579d999db4-kbs9l" event={"ID":"30316be2-7e96-4ccb-82b2-16332b3b848d","Type":"ContainerDied","Data":"86834a171ea51b8fde21eb54e28ee5bca60eda1ef84e0d51c18c87480539738f"} Jul 17 07:24:44.144653 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:44.144629 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579d999db4-kbs9l" event={"ID":"30316be2-7e96-4ccb-82b2-16332b3b848d","Type":"ContainerDied","Data":"19b649bb2f49357a576f135dfc37efab2c57a53fef03bb4d2da08ae6b3af4abb"} Jul 17 07:24:44.144766 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:44.144657 2568 scope.go:117] "RemoveContainer" containerID="86834a171ea51b8fde21eb54e28ee5bca60eda1ef84e0d51c18c87480539738f" Jul 17 07:24:44.153695 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:44.153669 2568 scope.go:117] "RemoveContainer" containerID="86834a171ea51b8fde21eb54e28ee5bca60eda1ef84e0d51c18c87480539738f" Jul 17 07:24:44.153973 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:24:44.153954 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86834a171ea51b8fde21eb54e28ee5bca60eda1ef84e0d51c18c87480539738f\": container with ID starting with 86834a171ea51b8fde21eb54e28ee5bca60eda1ef84e0d51c18c87480539738f not found: ID does not exist" containerID="86834a171ea51b8fde21eb54e28ee5bca60eda1ef84e0d51c18c87480539738f" Jul 17 07:24:44.154030 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:44.153982 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86834a171ea51b8fde21eb54e28ee5bca60eda1ef84e0d51c18c87480539738f"} err="failed to get container status \"86834a171ea51b8fde21eb54e28ee5bca60eda1ef84e0d51c18c87480539738f\": rpc error: code = NotFound desc = could not find container \"86834a171ea51b8fde21eb54e28ee5bca60eda1ef84e0d51c18c87480539738f\": container with ID starting with 86834a171ea51b8fde21eb54e28ee5bca60eda1ef84e0d51c18c87480539738f not found: ID does not exist" Jul 17 07:24:44.167517 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:44.167491 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-579d999db4-kbs9l"] Jul 17 07:24:44.169342 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:44.169320 2568 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/30316be2-7e96-4ccb-82b2-16332b3b848d-webhook-cert\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:24:44.169342 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:44.169340 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9zz4w\" (UniqueName: \"kubernetes.io/projected/30316be2-7e96-4ccb-82b2-16332b3b848d-kube-api-access-9zz4w\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:24:44.169901 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:44.169884 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-579d999db4-kbs9l"] Jul 17 07:24:44.969157 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:44.969119 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-579d999db4-94vrg"] Jul 17 07:24:44.969568 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:44.969536 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30316be2-7e96-4ccb-82b2-16332b3b848d" containerName="manager" Jul 17 07:24:44.969568 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:44.969548 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="30316be2-7e96-4ccb-82b2-16332b3b848d" containerName="manager" Jul 17 07:24:44.969661 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:44.969626 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="30316be2-7e96-4ccb-82b2-16332b3b848d" containerName="manager" Jul 17 07:24:44.974066 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:44.974045 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579d999db4-94vrg" Jul 17 07:24:44.977617 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:44.977598 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 17 07:24:44.977781 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:44.977757 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 17 07:24:44.978647 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:44.978630 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-x65gb\"" Jul 17 07:24:44.984562 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:44.984540 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-579d999db4-94vrg"] Jul 17 07:24:45.076057 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:45.076025 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/222416ad-c37f-4cc0-946f-7c11384a9f91-webhook-cert\") pod \"maas-controller-579d999db4-94vrg\" (UID: \"222416ad-c37f-4cc0-946f-7c11384a9f91\") " pod="opendatahub/maas-controller-579d999db4-94vrg" Jul 17 07:24:45.076177 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:45.076082 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbh6z\" (UniqueName: \"kubernetes.io/projected/222416ad-c37f-4cc0-946f-7c11384a9f91-kube-api-access-dbh6z\") pod \"maas-controller-579d999db4-94vrg\" (UID: \"222416ad-c37f-4cc0-946f-7c11384a9f91\") " pod="opendatahub/maas-controller-579d999db4-94vrg" Jul 17 07:24:45.177478 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:45.177451 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/222416ad-c37f-4cc0-946f-7c11384a9f91-webhook-cert\") pod \"maas-controller-579d999db4-94vrg\" (UID: \"222416ad-c37f-4cc0-946f-7c11384a9f91\") " pod="opendatahub/maas-controller-579d999db4-94vrg" Jul 17 07:24:45.177627 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:45.177495 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbh6z\" (UniqueName: \"kubernetes.io/projected/222416ad-c37f-4cc0-946f-7c11384a9f91-kube-api-access-dbh6z\") pod \"maas-controller-579d999db4-94vrg\" (UID: \"222416ad-c37f-4cc0-946f-7c11384a9f91\") " pod="opendatahub/maas-controller-579d999db4-94vrg" Jul 17 07:24:45.179781 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:45.179760 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/222416ad-c37f-4cc0-946f-7c11384a9f91-webhook-cert\") pod \"maas-controller-579d999db4-94vrg\" (UID: \"222416ad-c37f-4cc0-946f-7c11384a9f91\") " pod="opendatahub/maas-controller-579d999db4-94vrg" Jul 17 07:24:45.186327 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:45.186303 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbh6z\" (UniqueName: \"kubernetes.io/projected/222416ad-c37f-4cc0-946f-7c11384a9f91-kube-api-access-dbh6z\") pod \"maas-controller-579d999db4-94vrg\" (UID: \"222416ad-c37f-4cc0-946f-7c11384a9f91\") " pod="opendatahub/maas-controller-579d999db4-94vrg" Jul 17 07:24:45.285432 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:45.285356 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579d999db4-94vrg" Jul 17 07:24:45.422472 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:45.422445 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-579d999db4-94vrg"] Jul 17 07:24:45.423603 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:24:45.423577 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod222416ad_c37f_4cc0_946f_7c11384a9f91.slice/crio-4daffb59085e11e122599c55ddcf2637ec13141681d053d8b24bf628113da83e WatchSource:0}: Error finding container 4daffb59085e11e122599c55ddcf2637ec13141681d053d8b24bf628113da83e: Status 404 returned error can't find the container with id 4daffb59085e11e122599c55ddcf2637ec13141681d053d8b24bf628113da83e Jul 17 07:24:46.000840 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:46.000800 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30316be2-7e96-4ccb-82b2-16332b3b848d" path="/var/lib/kubelet/pods/30316be2-7e96-4ccb-82b2-16332b3b848d/volumes" Jul 17 07:24:46.158557 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:46.158517 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579d999db4-94vrg" event={"ID":"222416ad-c37f-4cc0-946f-7c11384a9f91","Type":"ContainerStarted","Data":"8d645fcbc22240f1bff087c8824eb2a65578cc9bbff2291df602ae3305f49f26"} Jul 17 07:24:46.158763 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:46.158575 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579d999db4-94vrg" event={"ID":"222416ad-c37f-4cc0-946f-7c11384a9f91","Type":"ContainerStarted","Data":"4daffb59085e11e122599c55ddcf2637ec13141681d053d8b24bf628113da83e"} Jul 17 07:24:46.158763 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:46.158598 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-579d999db4-94vrg" Jul 17 07:24:46.177344 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:46.177263 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-579d999db4-94vrg" podStartSLOduration=1.797120246 podStartE2EDuration="2.177244256s" totalImagesPullingTime="380.12401ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 07:24:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:24:46.176938601 +0000 UTC m=+790.661627578" watchObservedRunningTime="2026-07-17 07:24:46.177244256 +0000 UTC m=+790.661933230" Jul 17 07:24:57.165701 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:24:57.165662 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-579d999db4-94vrg" Jul 17 07:26:36.017166 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:26:36.017076 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-xr6pg_2ce00a54-15e9-42b7-b3a8-9e58d1101558/cluster-monitoring-operator/0.log" Jul 17 07:26:36.018937 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:26:36.018914 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-xr6pg_2ce00a54-15e9-42b7-b3a8-9e58d1101558/cluster-monitoring-operator/0.log" Jul 17 07:26:36.022279 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:26:36.022261 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-xpxv5_27538d1e-992b-4651-a2d2-7a7ecf442806/console-operator/1.log" Jul 17 07:26:36.024007 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:26:36.023988 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-xpxv5_27538d1e-992b-4651-a2d2-7a7ecf442806/console-operator/1.log" Jul 17 07:26:36.025491 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:26:36.025469 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cpwz4_1c44a3ae-4c83-4203-8b0b-116d6618accf/ovn-acl-logging/0.log" Jul 17 07:26:36.027424 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:26:36.027407 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cpwz4_1c44a3ae-4c83-4203-8b0b-116d6618accf/ovn-acl-logging/0.log" Jul 17 07:31:36.055199 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:31:36.055169 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-xr6pg_2ce00a54-15e9-42b7-b3a8-9e58d1101558/cluster-monitoring-operator/0.log" Jul 17 07:31:36.057597 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:31:36.056067 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-xr6pg_2ce00a54-15e9-42b7-b3a8-9e58d1101558/cluster-monitoring-operator/0.log" Jul 17 07:31:36.060330 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:31:36.060304 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-xpxv5_27538d1e-992b-4651-a2d2-7a7ecf442806/console-operator/1.log" Jul 17 07:31:36.060990 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:31:36.060972 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-xpxv5_27538d1e-992b-4651-a2d2-7a7ecf442806/console-operator/1.log" Jul 17 07:31:36.063663 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:31:36.063644 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cpwz4_1c44a3ae-4c83-4203-8b0b-116d6618accf/ovn-acl-logging/0.log" Jul 17 07:31:36.064647 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:31:36.064630 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cpwz4_1c44a3ae-4c83-4203-8b0b-116d6618accf/ovn-acl-logging/0.log" Jul 17 07:36:36.090982 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:36.090877 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-xr6pg_2ce00a54-15e9-42b7-b3a8-9e58d1101558/cluster-monitoring-operator/0.log" Jul 17 07:36:36.095101 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:36.093582 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-xr6pg_2ce00a54-15e9-42b7-b3a8-9e58d1101558/cluster-monitoring-operator/0.log" Jul 17 07:36:36.095864 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:36.095835 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-xpxv5_27538d1e-992b-4651-a2d2-7a7ecf442806/console-operator/1.log" Jul 17 07:36:36.098703 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:36.098682 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-xpxv5_27538d1e-992b-4651-a2d2-7a7ecf442806/console-operator/1.log" Jul 17 07:36:36.099364 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:36.099350 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cpwz4_1c44a3ae-4c83-4203-8b0b-116d6618accf/ovn-acl-logging/0.log" Jul 17 07:36:36.102551 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:36.102534 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cpwz4_1c44a3ae-4c83-4203-8b0b-116d6618accf/ovn-acl-logging/0.log" Jul 17 07:36:41.815696 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:41.815666 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f"] Jul 17 07:36:41.819241 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:41.819219 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f" Jul 17 07:36:41.821743 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:41.821720 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-2dfadf83-openshift-default-dockercfg-c6fv4\"" Jul 17 07:36:41.836207 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:41.836179 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f"] Jul 17 07:36:41.917997 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:41.917952 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/80c86118-1a2e-4064-93c1-43c513d06dca-istio-envoy\") pod \"e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f\" (UID: \"80c86118-1a2e-4064-93c1-43c513d06dca\") " pod="openshift-ingress/e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f" Jul 17 07:36:41.917997 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:41.918004 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/80c86118-1a2e-4064-93c1-43c513d06dca-istio-data\") pod \"e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f\" (UID: \"80c86118-1a2e-4064-93c1-43c513d06dca\") " pod="openshift-ingress/e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f" Jul 17 07:36:41.918208 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:41.918027 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/80c86118-1a2e-4064-93c1-43c513d06dca-workload-socket\") pod \"e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f\" (UID: \"80c86118-1a2e-4064-93c1-43c513d06dca\") " pod="openshift-ingress/e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f" Jul 17 07:36:41.918208 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:41.918068 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rd9j\" (UniqueName: \"kubernetes.io/projected/80c86118-1a2e-4064-93c1-43c513d06dca-kube-api-access-9rd9j\") pod \"e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f\" (UID: \"80c86118-1a2e-4064-93c1-43c513d06dca\") " pod="openshift-ingress/e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f" Jul 17 07:36:41.918208 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:41.918096 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/80c86118-1a2e-4064-93c1-43c513d06dca-workload-certs\") pod \"e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f\" (UID: \"80c86118-1a2e-4064-93c1-43c513d06dca\") " pod="openshift-ingress/e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f" Jul 17 07:36:41.918208 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:41.918182 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/80c86118-1a2e-4064-93c1-43c513d06dca-istio-token\") pod \"e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f\" (UID: \"80c86118-1a2e-4064-93c1-43c513d06dca\") " pod="openshift-ingress/e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f" Jul 17 07:36:41.918365 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:41.918228 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/80c86118-1a2e-4064-93c1-43c513d06dca-istio-podinfo\") pod \"e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f\" (UID: \"80c86118-1a2e-4064-93c1-43c513d06dca\") " pod="openshift-ingress/e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f" Jul 17 07:36:41.918365 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:41.918254 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/80c86118-1a2e-4064-93c1-43c513d06dca-credential-socket\") pod \"e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f\" (UID: \"80c86118-1a2e-4064-93c1-43c513d06dca\") " pod="openshift-ingress/e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f" Jul 17 07:36:41.918365 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:41.918271 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/80c86118-1a2e-4064-93c1-43c513d06dca-istiod-ca-cert\") pod \"e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f\" (UID: \"80c86118-1a2e-4064-93c1-43c513d06dca\") " pod="openshift-ingress/e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f" Jul 17 07:36:42.019344 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:42.019273 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rd9j\" (UniqueName: \"kubernetes.io/projected/80c86118-1a2e-4064-93c1-43c513d06dca-kube-api-access-9rd9j\") pod \"e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f\" (UID: \"80c86118-1a2e-4064-93c1-43c513d06dca\") " pod="openshift-ingress/e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f" Jul 17 07:36:42.019344 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:42.019352 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/80c86118-1a2e-4064-93c1-43c513d06dca-workload-certs\") pod \"e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f\" (UID: \"80c86118-1a2e-4064-93c1-43c513d06dca\") " pod="openshift-ingress/e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f" Jul 17 07:36:42.019668 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:42.019436 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/80c86118-1a2e-4064-93c1-43c513d06dca-istio-token\") pod \"e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f\" (UID: \"80c86118-1a2e-4064-93c1-43c513d06dca\") " pod="openshift-ingress/e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f" Jul 17 07:36:42.019668 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:42.019474 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/80c86118-1a2e-4064-93c1-43c513d06dca-istio-podinfo\") pod \"e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f\" (UID: \"80c86118-1a2e-4064-93c1-43c513d06dca\") " pod="openshift-ingress/e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f" Jul 17 07:36:42.019668 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:42.019507 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/80c86118-1a2e-4064-93c1-43c513d06dca-credential-socket\") pod \"e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f\" (UID: \"80c86118-1a2e-4064-93c1-43c513d06dca\") " pod="openshift-ingress/e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f" Jul 17 07:36:42.019843 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:42.019647 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/80c86118-1a2e-4064-93c1-43c513d06dca-istiod-ca-cert\") pod \"e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f\" (UID: \"80c86118-1a2e-4064-93c1-43c513d06dca\") " pod="openshift-ingress/e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f" Jul 17 07:36:42.019843 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:42.019713 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/80c86118-1a2e-4064-93c1-43c513d06dca-workload-certs\") pod \"e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f\" (UID: \"80c86118-1a2e-4064-93c1-43c513d06dca\") " pod="openshift-ingress/e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f" Jul 17 07:36:42.019843 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:42.019777 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/80c86118-1a2e-4064-93c1-43c513d06dca-istio-envoy\") pod \"e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f\" (UID: \"80c86118-1a2e-4064-93c1-43c513d06dca\") " pod="openshift-ingress/e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f" Jul 17 07:36:42.019843 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:42.019800 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/80c86118-1a2e-4064-93c1-43c513d06dca-istio-data\") pod \"e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f\" (UID: \"80c86118-1a2e-4064-93c1-43c513d06dca\") " pod="openshift-ingress/e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f" Jul 17 07:36:42.019843 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:42.019820 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/80c86118-1a2e-4064-93c1-43c513d06dca-workload-socket\") pod \"e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f\" (UID: \"80c86118-1a2e-4064-93c1-43c513d06dca\") " pod="openshift-ingress/e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f" Jul 17 07:36:42.019843 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:42.019832 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/80c86118-1a2e-4064-93c1-43c513d06dca-credential-socket\") pod \"e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f\" (UID: \"80c86118-1a2e-4064-93c1-43c513d06dca\") " pod="openshift-ingress/e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f" Jul 17 07:36:42.020178 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:42.020154 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/80c86118-1a2e-4064-93c1-43c513d06dca-istio-data\") pod \"e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f\" (UID: \"80c86118-1a2e-4064-93c1-43c513d06dca\") " pod="openshift-ingress/e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f" Jul 17 07:36:42.020307 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:42.020260 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/80c86118-1a2e-4064-93c1-43c513d06dca-workload-socket\") pod \"e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f\" (UID: \"80c86118-1a2e-4064-93c1-43c513d06dca\") " pod="openshift-ingress/e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f" Jul 17 07:36:42.020369 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:42.020318 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/80c86118-1a2e-4064-93c1-43c513d06dca-istiod-ca-cert\") pod \"e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f\" (UID: \"80c86118-1a2e-4064-93c1-43c513d06dca\") " pod="openshift-ingress/e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f" Jul 17 07:36:42.021850 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:42.021828 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/80c86118-1a2e-4064-93c1-43c513d06dca-istio-envoy\") pod \"e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f\" (UID: \"80c86118-1a2e-4064-93c1-43c513d06dca\") " pod="openshift-ingress/e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f" Jul 17 07:36:42.022072 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:42.022056 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/80c86118-1a2e-4064-93c1-43c513d06dca-istio-podinfo\") pod \"e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f\" (UID: \"80c86118-1a2e-4064-93c1-43c513d06dca\") " pod="openshift-ingress/e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f" Jul 17 07:36:42.027958 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:42.027937 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/80c86118-1a2e-4064-93c1-43c513d06dca-istio-token\") pod \"e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f\" (UID: \"80c86118-1a2e-4064-93c1-43c513d06dca\") " pod="openshift-ingress/e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f" Jul 17 07:36:42.028353 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:42.028329 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rd9j\" (UniqueName: \"kubernetes.io/projected/80c86118-1a2e-4064-93c1-43c513d06dca-kube-api-access-9rd9j\") pod \"e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f\" (UID: \"80c86118-1a2e-4064-93c1-43c513d06dca\") " pod="openshift-ingress/e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f" Jul 17 07:36:42.130039 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:42.130006 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f" Jul 17 07:36:42.268351 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:42.268317 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f"] Jul 17 07:36:42.269456 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:36:42.269428 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80c86118_1a2e_4064_93c1_43c513d06dca.slice/crio-453946ba060efc11531b21ab7e4374aea8e8009063c3c7d9873b1e3bbbd1a9bd WatchSource:0}: Error finding container 453946ba060efc11531b21ab7e4374aea8e8009063c3c7d9873b1e3bbbd1a9bd: Status 404 returned error can't find the container with id 453946ba060efc11531b21ab7e4374aea8e8009063c3c7d9873b1e3bbbd1a9bd Jul 17 07:36:42.272195 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:42.272154 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:36:42.272272 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:42.272237 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:36:42.272361 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:42.272277 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:36:43.050462 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:43.050422 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f" event={"ID":"80c86118-1a2e-4064-93c1-43c513d06dca","Type":"ContainerStarted","Data":"ed74ce46e22953de823385435e7b65baa37038ea4f69a86463fe2bc8dbfb1cb8"} Jul 17 07:36:43.050936 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:43.050474 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f" event={"ID":"80c86118-1a2e-4064-93c1-43c513d06dca","Type":"ContainerStarted","Data":"453946ba060efc11531b21ab7e4374aea8e8009063c3c7d9873b1e3bbbd1a9bd"} Jul 17 07:36:43.073523 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:43.073469 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f" podStartSLOduration=2.073452921 podStartE2EDuration="2.073452921s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 07:36:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:36:43.071232777 +0000 UTC m=+1507.555921753" watchObservedRunningTime="2026-07-17 07:36:43.073452921 +0000 UTC m=+1507.558141894" Jul 17 07:36:43.130439 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:43.130404 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f" Jul 17 07:36:43.135484 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:43.135461 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f" Jul 17 07:36:44.057128 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:44.057092 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f" Jul 17 07:36:44.058042 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:44.058001 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f" Jul 17 07:36:54.130585 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:54.130553 2568 generic.go:356] "Generic (PLEG): container finished" podID="80c86118-1a2e-4064-93c1-43c513d06dca" containerID="ed74ce46e22953de823385435e7b65baa37038ea4f69a86463fe2bc8dbfb1cb8" exitCode=0 Jul 17 07:36:54.130999 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:54.130598 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f" event={"ID":"80c86118-1a2e-4064-93c1-43c513d06dca","Type":"ContainerDied","Data":"ed74ce46e22953de823385435e7b65baa37038ea4f69a86463fe2bc8dbfb1cb8"} Jul 17 07:36:54.130999 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:54.130954 2568 scope.go:117] "RemoveContainer" containerID="ed74ce46e22953de823385435e7b65baa37038ea4f69a86463fe2bc8dbfb1cb8" Jul 17 07:36:54.131991 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:54.131957 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:36:54.132063 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:54.132046 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:36:54.132101 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:54.132092 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:36:55.139063 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:55.139027 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f" event={"ID":"80c86118-1a2e-4064-93c1-43c513d06dca","Type":"ContainerStarted","Data":"5e876f5c066dc865753f3234c82ed3ec33044c5e4d6e931894cdb8ef05e35966"} Jul 17 07:36:56.130708 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:56.130675 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f" Jul 17 07:36:56.131889 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:56.131860 2568 patch_prober.go:28] interesting pod/e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 17 07:36:56.132031 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:56.131931 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f" podUID="80c86118-1a2e-4064-93c1-43c513d06dca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 17 07:36:57.131344 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:57.131278 2568 patch_prober.go:28] interesting pod/e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 17 07:36:57.131830 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:57.131365 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f" podUID="80c86118-1a2e-4064-93c1-43c513d06dca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 17 07:36:58.130635 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:58.130599 2568 patch_prober.go:28] interesting pod/e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 17 07:36:58.130843 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:58.130660 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f" podUID="80c86118-1a2e-4064-93c1-43c513d06dca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 17 07:36:59.057700 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:59.057669 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f" Jul 17 07:36:59.134638 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:59.134612 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f" Jul 17 07:36:59.167555 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:36:59.167527 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f" Jul 17 07:39:48.163007 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:48.162970 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f"] Jul 17 07:39:48.163576 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:48.163200 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f" podUID="80c86118-1a2e-4064-93c1-43c513d06dca" containerName="istio-proxy" containerID="cri-o://5e876f5c066dc865753f3234c82ed3ec33044c5e4d6e931894cdb8ef05e35966" gracePeriod=30 Jul 17 07:39:53.414087 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:53.414062 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f" Jul 17 07:39:53.450570 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:53.450495 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/80c86118-1a2e-4064-93c1-43c513d06dca-istio-podinfo\") pod \"80c86118-1a2e-4064-93c1-43c513d06dca\" (UID: \"80c86118-1a2e-4064-93c1-43c513d06dca\") " Jul 17 07:39:53.450570 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:53.450549 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/80c86118-1a2e-4064-93c1-43c513d06dca-istio-envoy\") pod \"80c86118-1a2e-4064-93c1-43c513d06dca\" (UID: \"80c86118-1a2e-4064-93c1-43c513d06dca\") " Jul 17 07:39:53.450783 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:53.450600 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/80c86118-1a2e-4064-93c1-43c513d06dca-istio-data\") pod \"80c86118-1a2e-4064-93c1-43c513d06dca\" (UID: \"80c86118-1a2e-4064-93c1-43c513d06dca\") " Jul 17 07:39:53.450783 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:53.450622 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9rd9j\" (UniqueName: \"kubernetes.io/projected/80c86118-1a2e-4064-93c1-43c513d06dca-kube-api-access-9rd9j\") pod \"80c86118-1a2e-4064-93c1-43c513d06dca\" (UID: \"80c86118-1a2e-4064-93c1-43c513d06dca\") " Jul 17 07:39:53.450783 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:53.450696 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/80c86118-1a2e-4064-93c1-43c513d06dca-workload-certs\") pod \"80c86118-1a2e-4064-93c1-43c513d06dca\" (UID: \"80c86118-1a2e-4064-93c1-43c513d06dca\") " Jul 17 07:39:53.450783 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:53.450722 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/80c86118-1a2e-4064-93c1-43c513d06dca-istio-token\") pod \"80c86118-1a2e-4064-93c1-43c513d06dca\" (UID: \"80c86118-1a2e-4064-93c1-43c513d06dca\") " Jul 17 07:39:53.450783 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:53.450755 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/80c86118-1a2e-4064-93c1-43c513d06dca-workload-socket\") pod \"80c86118-1a2e-4064-93c1-43c513d06dca\" (UID: \"80c86118-1a2e-4064-93c1-43c513d06dca\") " Jul 17 07:39:53.450783 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:53.450780 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/80c86118-1a2e-4064-93c1-43c513d06dca-istiod-ca-cert\") pod \"80c86118-1a2e-4064-93c1-43c513d06dca\" (UID: \"80c86118-1a2e-4064-93c1-43c513d06dca\") " Jul 17 07:39:53.451111 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:53.450808 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/80c86118-1a2e-4064-93c1-43c513d06dca-credential-socket\") pod \"80c86118-1a2e-4064-93c1-43c513d06dca\" (UID: \"80c86118-1a2e-4064-93c1-43c513d06dca\") " Jul 17 07:39:53.451595 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:53.451429 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80c86118-1a2e-4064-93c1-43c513d06dca-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "80c86118-1a2e-4064-93c1-43c513d06dca" (UID: "80c86118-1a2e-4064-93c1-43c513d06dca"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:39:53.451743 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:53.451612 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80c86118-1a2e-4064-93c1-43c513d06dca-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "80c86118-1a2e-4064-93c1-43c513d06dca" (UID: "80c86118-1a2e-4064-93c1-43c513d06dca"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 07:39:53.451743 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:53.451483 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80c86118-1a2e-4064-93c1-43c513d06dca-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "80c86118-1a2e-4064-93c1-43c513d06dca" (UID: "80c86118-1a2e-4064-93c1-43c513d06dca"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:39:53.451743 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:53.451519 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80c86118-1a2e-4064-93c1-43c513d06dca-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "80c86118-1a2e-4064-93c1-43c513d06dca" (UID: "80c86118-1a2e-4064-93c1-43c513d06dca"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:39:53.451743 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:53.451636 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80c86118-1a2e-4064-93c1-43c513d06dca-istio-data" (OuterVolumeSpecName: "istio-data") pod "80c86118-1a2e-4064-93c1-43c513d06dca" (UID: "80c86118-1a2e-4064-93c1-43c513d06dca"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:39:53.453320 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:53.453254 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80c86118-1a2e-4064-93c1-43c513d06dca-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "80c86118-1a2e-4064-93c1-43c513d06dca" (UID: "80c86118-1a2e-4064-93c1-43c513d06dca"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:39:53.453456 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:53.453322 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80c86118-1a2e-4064-93c1-43c513d06dca-kube-api-access-9rd9j" (OuterVolumeSpecName: "kube-api-access-9rd9j") pod "80c86118-1a2e-4064-93c1-43c513d06dca" (UID: "80c86118-1a2e-4064-93c1-43c513d06dca"). InnerVolumeSpecName "kube-api-access-9rd9j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 07:39:53.453942 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:53.453869 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/80c86118-1a2e-4064-93c1-43c513d06dca-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "80c86118-1a2e-4064-93c1-43c513d06dca" (UID: "80c86118-1a2e-4064-93c1-43c513d06dca"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 17 07:39:53.454644 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:53.454617 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80c86118-1a2e-4064-93c1-43c513d06dca-istio-token" (OuterVolumeSpecName: "istio-token") pod "80c86118-1a2e-4064-93c1-43c513d06dca" (UID: "80c86118-1a2e-4064-93c1-43c513d06dca"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 07:39:53.552250 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:53.552213 2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/80c86118-1a2e-4064-93c1-43c513d06dca-workload-certs\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:39:53.552250 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:53.552242 2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/80c86118-1a2e-4064-93c1-43c513d06dca-istio-token\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:39:53.552250 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:53.552251 2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/80c86118-1a2e-4064-93c1-43c513d06dca-workload-socket\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:39:53.552470 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:53.552259 2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/80c86118-1a2e-4064-93c1-43c513d06dca-istiod-ca-cert\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:39:53.552470 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:53.552267 2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/80c86118-1a2e-4064-93c1-43c513d06dca-credential-socket\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:39:53.552470 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:53.552278 2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/80c86118-1a2e-4064-93c1-43c513d06dca-istio-podinfo\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:39:53.552470 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:53.552310 2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/80c86118-1a2e-4064-93c1-43c513d06dca-istio-envoy\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:39:53.552470 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:53.552317 2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/80c86118-1a2e-4064-93c1-43c513d06dca-istio-data\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:39:53.552470 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:53.552325 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9rd9j\" (UniqueName: \"kubernetes.io/projected/80c86118-1a2e-4064-93c1-43c513d06dca-kube-api-access-9rd9j\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:39:54.125253 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:54.125212 2568 generic.go:356] "Generic (PLEG): container finished" podID="80c86118-1a2e-4064-93c1-43c513d06dca" containerID="5e876f5c066dc865753f3234c82ed3ec33044c5e4d6e931894cdb8ef05e35966" exitCode=0 Jul 17 07:39:54.125484 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:54.125318 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f" event={"ID":"80c86118-1a2e-4064-93c1-43c513d06dca","Type":"ContainerDied","Data":"5e876f5c066dc865753f3234c82ed3ec33044c5e4d6e931894cdb8ef05e35966"} Jul 17 07:39:54.125484 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:54.125346 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f" Jul 17 07:39:54.125484 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:54.125383 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f" event={"ID":"80c86118-1a2e-4064-93c1-43c513d06dca","Type":"ContainerDied","Data":"453946ba060efc11531b21ab7e4374aea8e8009063c3c7d9873b1e3bbbd1a9bd"} Jul 17 07:39:54.125484 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:54.125414 2568 scope.go:117] "RemoveContainer" containerID="5e876f5c066dc865753f3234c82ed3ec33044c5e4d6e931894cdb8ef05e35966" Jul 17 07:39:54.135723 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:54.135703 2568 scope.go:117] "RemoveContainer" containerID="ed74ce46e22953de823385435e7b65baa37038ea4f69a86463fe2bc8dbfb1cb8" Jul 17 07:39:54.143953 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:54.143935 2568 scope.go:117] "RemoveContainer" containerID="5e876f5c066dc865753f3234c82ed3ec33044c5e4d6e931894cdb8ef05e35966" Jul 17 07:39:54.144212 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:39:54.144193 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e876f5c066dc865753f3234c82ed3ec33044c5e4d6e931894cdb8ef05e35966\": container with ID starting with 5e876f5c066dc865753f3234c82ed3ec33044c5e4d6e931894cdb8ef05e35966 not found: ID does not exist" containerID="5e876f5c066dc865753f3234c82ed3ec33044c5e4d6e931894cdb8ef05e35966" Jul 17 07:39:54.144294 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:54.144221 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e876f5c066dc865753f3234c82ed3ec33044c5e4d6e931894cdb8ef05e35966"} err="failed to get container status \"5e876f5c066dc865753f3234c82ed3ec33044c5e4d6e931894cdb8ef05e35966\": rpc error: code = NotFound desc = could not find container \"5e876f5c066dc865753f3234c82ed3ec33044c5e4d6e931894cdb8ef05e35966\": container with ID starting with 5e876f5c066dc865753f3234c82ed3ec33044c5e4d6e931894cdb8ef05e35966 not found: ID does not exist" Jul 17 07:39:54.144294 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:54.144238 2568 scope.go:117] "RemoveContainer" containerID="ed74ce46e22953de823385435e7b65baa37038ea4f69a86463fe2bc8dbfb1cb8" Jul 17 07:39:54.144523 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:39:54.144502 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed74ce46e22953de823385435e7b65baa37038ea4f69a86463fe2bc8dbfb1cb8\": container with ID starting with ed74ce46e22953de823385435e7b65baa37038ea4f69a86463fe2bc8dbfb1cb8 not found: ID does not exist" containerID="ed74ce46e22953de823385435e7b65baa37038ea4f69a86463fe2bc8dbfb1cb8" Jul 17 07:39:54.144583 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:54.144534 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed74ce46e22953de823385435e7b65baa37038ea4f69a86463fe2bc8dbfb1cb8"} err="failed to get container status \"ed74ce46e22953de823385435e7b65baa37038ea4f69a86463fe2bc8dbfb1cb8\": rpc error: code = NotFound desc = could not find container \"ed74ce46e22953de823385435e7b65baa37038ea4f69a86463fe2bc8dbfb1cb8\": container with ID starting with ed74ce46e22953de823385435e7b65baa37038ea4f69a86463fe2bc8dbfb1cb8 not found: ID does not exist" Jul 17 07:39:54.145199 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:54.145181 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f"] Jul 17 07:39:54.150933 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:54.150911 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-2dfadf83-openshift-default-66bf969759-bz69f"] Jul 17 07:39:55.215170 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:55.215138 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466"] Jul 17 07:39:55.216219 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:55.216194 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80c86118-1a2e-4064-93c1-43c513d06dca" containerName="istio-proxy" Jul 17 07:39:55.216219 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:55.216220 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="80c86118-1a2e-4064-93c1-43c513d06dca" containerName="istio-proxy" Jul 17 07:39:55.216381 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:55.216231 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80c86118-1a2e-4064-93c1-43c513d06dca" containerName="istio-proxy" Jul 17 07:39:55.216381 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:55.216240 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="80c86118-1a2e-4064-93c1-43c513d06dca" containerName="istio-proxy" Jul 17 07:39:55.216381 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:55.216346 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="80c86118-1a2e-4064-93c1-43c513d06dca" containerName="istio-proxy" Jul 17 07:39:55.216381 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:55.216356 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="80c86118-1a2e-4064-93c1-43c513d06dca" containerName="istio-proxy" Jul 17 07:39:55.221332 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:55.221313 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466" Jul 17 07:39:55.224795 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:55.224777 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-b531d224-gw-openshift-default-dockercfg-x92nb\"" Jul 17 07:39:55.237736 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:55.237715 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466"] Jul 17 07:39:55.266480 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:55.266450 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7z622\" (UniqueName: \"kubernetes.io/projected/772de92b-4481-406c-813d-908b0926d9eb-kube-api-access-7z622\") pod \"e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466\" (UID: \"772de92b-4481-406c-813d-908b0926d9eb\") " pod="openshift-ingress/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466" Jul 17 07:39:55.266616 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:55.266557 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/772de92b-4481-406c-813d-908b0926d9eb-credential-socket\") pod \"e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466\" (UID: \"772de92b-4481-406c-813d-908b0926d9eb\") " pod="openshift-ingress/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466" Jul 17 07:39:55.266616 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:55.266597 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/772de92b-4481-406c-813d-908b0926d9eb-istio-envoy\") pod \"e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466\" (UID: \"772de92b-4481-406c-813d-908b0926d9eb\") " pod="openshift-ingress/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466" Jul 17 07:39:55.266701 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:55.266650 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/772de92b-4481-406c-813d-908b0926d9eb-istio-podinfo\") pod \"e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466\" (UID: \"772de92b-4481-406c-813d-908b0926d9eb\") " pod="openshift-ingress/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466" Jul 17 07:39:55.266701 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:55.266679 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/772de92b-4481-406c-813d-908b0926d9eb-workload-socket\") pod \"e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466\" (UID: \"772de92b-4481-406c-813d-908b0926d9eb\") " pod="openshift-ingress/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466" Jul 17 07:39:55.266766 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:55.266702 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/772de92b-4481-406c-813d-908b0926d9eb-istio-data\") pod \"e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466\" (UID: \"772de92b-4481-406c-813d-908b0926d9eb\") " pod="openshift-ingress/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466" Jul 17 07:39:55.266766 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:55.266728 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/772de92b-4481-406c-813d-908b0926d9eb-workload-certs\") pod \"e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466\" (UID: \"772de92b-4481-406c-813d-908b0926d9eb\") " pod="openshift-ingress/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466" Jul 17 07:39:55.266766 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:55.266754 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/772de92b-4481-406c-813d-908b0926d9eb-istiod-ca-cert\") pod \"e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466\" (UID: \"772de92b-4481-406c-813d-908b0926d9eb\") " pod="openshift-ingress/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466" Jul 17 07:39:55.266869 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:55.266839 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/772de92b-4481-406c-813d-908b0926d9eb-istio-token\") pod \"e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466\" (UID: \"772de92b-4481-406c-813d-908b0926d9eb\") " pod="openshift-ingress/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466" Jul 17 07:39:55.367604 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:55.367574 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/772de92b-4481-406c-813d-908b0926d9eb-istio-envoy\") pod \"e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466\" (UID: \"772de92b-4481-406c-813d-908b0926d9eb\") " pod="openshift-ingress/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466" Jul 17 07:39:55.367778 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:55.367628 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/772de92b-4481-406c-813d-908b0926d9eb-istio-podinfo\") pod \"e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466\" (UID: \"772de92b-4481-406c-813d-908b0926d9eb\") " pod="openshift-ingress/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466" Jul 17 07:39:55.367778 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:55.367657 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/772de92b-4481-406c-813d-908b0926d9eb-workload-socket\") pod \"e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466\" (UID: \"772de92b-4481-406c-813d-908b0926d9eb\") " pod="openshift-ingress/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466" Jul 17 07:39:55.367778 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:55.367673 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/772de92b-4481-406c-813d-908b0926d9eb-istio-data\") pod \"e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466\" (UID: \"772de92b-4481-406c-813d-908b0926d9eb\") " pod="openshift-ingress/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466" Jul 17 07:39:55.367778 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:55.367699 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/772de92b-4481-406c-813d-908b0926d9eb-workload-certs\") pod \"e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466\" (UID: \"772de92b-4481-406c-813d-908b0926d9eb\") " pod="openshift-ingress/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466" Jul 17 07:39:55.367778 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:55.367723 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/772de92b-4481-406c-813d-908b0926d9eb-istiod-ca-cert\") pod \"e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466\" (UID: \"772de92b-4481-406c-813d-908b0926d9eb\") " pod="openshift-ingress/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466" Jul 17 07:39:55.367778 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:55.367758 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/772de92b-4481-406c-813d-908b0926d9eb-istio-token\") pod \"e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466\" (UID: \"772de92b-4481-406c-813d-908b0926d9eb\") " pod="openshift-ingress/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466" Jul 17 07:39:55.368161 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:55.367823 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7z622\" (UniqueName: \"kubernetes.io/projected/772de92b-4481-406c-813d-908b0926d9eb-kube-api-access-7z622\") pod \"e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466\" (UID: \"772de92b-4481-406c-813d-908b0926d9eb\") " pod="openshift-ingress/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466" Jul 17 07:39:55.368161 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:55.367889 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/772de92b-4481-406c-813d-908b0926d9eb-credential-socket\") pod \"e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466\" (UID: \"772de92b-4481-406c-813d-908b0926d9eb\") " pod="openshift-ingress/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466" Jul 17 07:39:55.368161 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:55.368043 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/772de92b-4481-406c-813d-908b0926d9eb-workload-socket\") pod \"e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466\" (UID: \"772de92b-4481-406c-813d-908b0926d9eb\") " pod="openshift-ingress/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466" Jul 17 07:39:55.368161 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:55.368141 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/772de92b-4481-406c-813d-908b0926d9eb-istio-data\") pod \"e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466\" (UID: \"772de92b-4481-406c-813d-908b0926d9eb\") " pod="openshift-ingress/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466" Jul 17 07:39:55.368356 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:55.368258 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/772de92b-4481-406c-813d-908b0926d9eb-credential-socket\") pod \"e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466\" (UID: \"772de92b-4481-406c-813d-908b0926d9eb\") " pod="openshift-ingress/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466" Jul 17 07:39:55.368411 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:55.368390 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/772de92b-4481-406c-813d-908b0926d9eb-workload-certs\") pod \"e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466\" (UID: \"772de92b-4481-406c-813d-908b0926d9eb\") " pod="openshift-ingress/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466" Jul 17 07:39:55.368752 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:55.368734 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/772de92b-4481-406c-813d-908b0926d9eb-istiod-ca-cert\") pod \"e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466\" (UID: \"772de92b-4481-406c-813d-908b0926d9eb\") " pod="openshift-ingress/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466" Jul 17 07:39:55.369883 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:55.369866 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/772de92b-4481-406c-813d-908b0926d9eb-istio-envoy\") pod \"e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466\" (UID: \"772de92b-4481-406c-813d-908b0926d9eb\") " pod="openshift-ingress/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466" Jul 17 07:39:55.370561 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:55.370538 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/772de92b-4481-406c-813d-908b0926d9eb-istio-podinfo\") pod \"e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466\" (UID: \"772de92b-4481-406c-813d-908b0926d9eb\") " pod="openshift-ingress/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466" Jul 17 07:39:55.376730 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:55.376701 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/772de92b-4481-406c-813d-908b0926d9eb-istio-token\") pod \"e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466\" (UID: \"772de92b-4481-406c-813d-908b0926d9eb\") " pod="openshift-ingress/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466" Jul 17 07:39:55.377226 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:55.377209 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7z622\" (UniqueName: \"kubernetes.io/projected/772de92b-4481-406c-813d-908b0926d9eb-kube-api-access-7z622\") pod \"e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466\" (UID: \"772de92b-4481-406c-813d-908b0926d9eb\") " pod="openshift-ingress/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466" Jul 17 07:39:55.534358 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:55.534322 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466" Jul 17 07:39:55.679649 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:55.679619 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466"] Jul 17 07:39:55.680307 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:39:55.680266 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod772de92b_4481_406c_813d_908b0926d9eb.slice/crio-4444b229e35f4b925ad8f3e381fce3170e87f9427b286e3a88c7d40d22caf759 WatchSource:0}: Error finding container 4444b229e35f4b925ad8f3e381fce3170e87f9427b286e3a88c7d40d22caf759: Status 404 returned error can't find the container with id 4444b229e35f4b925ad8f3e381fce3170e87f9427b286e3a88c7d40d22caf759 Jul 17 07:39:55.682365 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:55.682333 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:39:55.682461 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:55.682403 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:39:55.682461 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:55.682435 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:39:55.999873 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:55.999791 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80c86118-1a2e-4064-93c1-43c513d06dca" path="/var/lib/kubelet/pods/80c86118-1a2e-4064-93c1-43c513d06dca/volumes" Jul 17 07:39:56.138878 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:56.138837 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466" event={"ID":"772de92b-4481-406c-813d-908b0926d9eb","Type":"ContainerStarted","Data":"98aea4dc4b5e8e21f96b239ae0560879c7fab4f4ff5d6c45ac5987d2385e4be9"} Jul 17 07:39:56.139027 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:56.138902 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466" event={"ID":"772de92b-4481-406c-813d-908b0926d9eb","Type":"ContainerStarted","Data":"4444b229e35f4b925ad8f3e381fce3170e87f9427b286e3a88c7d40d22caf759"} Jul 17 07:39:56.161925 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:56.161874 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466" podStartSLOduration=1.161861165 podStartE2EDuration="1.161861165s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 07:39:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:39:56.160532125 +0000 UTC m=+1700.645221101" watchObservedRunningTime="2026-07-17 07:39:56.161861165 +0000 UTC m=+1700.646550208" Jul 17 07:39:56.534872 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:56.534832 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466" Jul 17 07:39:56.539547 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:56.539519 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466" Jul 17 07:39:57.144652 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:57.144625 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466" Jul 17 07:39:57.145764 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:39:57.145745 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466" Jul 17 07:40:11.229317 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:40:11.229263 2568 generic.go:356] "Generic (PLEG): container finished" podID="772de92b-4481-406c-813d-908b0926d9eb" containerID="98aea4dc4b5e8e21f96b239ae0560879c7fab4f4ff5d6c45ac5987d2385e4be9" exitCode=0 Jul 17 07:40:11.229739 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:40:11.229335 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466" event={"ID":"772de92b-4481-406c-813d-908b0926d9eb","Type":"ContainerDied","Data":"98aea4dc4b5e8e21f96b239ae0560879c7fab4f4ff5d6c45ac5987d2385e4be9"} Jul 17 07:40:11.229739 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:40:11.229705 2568 scope.go:117] "RemoveContainer" containerID="98aea4dc4b5e8e21f96b239ae0560879c7fab4f4ff5d6c45ac5987d2385e4be9" Jul 17 07:40:11.230657 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:40:11.230629 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:40:11.230717 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:40:11.230694 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:40:11.230753 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:40:11.230726 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:40:12.145365 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:40:12.145334 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466" Jul 17 07:40:12.236465 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:40:12.236425 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466" event={"ID":"772de92b-4481-406c-813d-908b0926d9eb","Type":"ContainerStarted","Data":"350675981c495504c3475292daf386ac3c5f4a374c6fe562ad95c7f69a101d95"} Jul 17 07:40:12.535418 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:40:12.535320 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466" Jul 17 07:40:12.536271 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:40:12.536249 2568 patch_prober.go:28] interesting pod/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 17 07:40:12.536349 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:40:12.536316 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466" podUID="772de92b-4481-406c-813d-908b0926d9eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 17 07:40:13.535160 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:40:13.535126 2568 patch_prober.go:28] interesting pod/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 17 07:40:13.535652 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:40:13.535189 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466" podUID="772de92b-4481-406c-813d-908b0926d9eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 17 07:40:14.535293 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:40:14.535250 2568 patch_prober.go:28] interesting pod/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 17 07:40:14.535655 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:40:14.535320 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466" podUID="772de92b-4481-406c-813d-908b0926d9eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 17 07:40:15.535090 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:40:15.535047 2568 patch_prober.go:28] interesting pod/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 17 07:40:15.535339 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:40:15.535110 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466" podUID="772de92b-4481-406c-813d-908b0926d9eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 17 07:40:16.539063 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:40:16.539033 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466" Jul 17 07:40:16.539536 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:40:16.539313 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466" Jul 17 07:40:16.540064 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:40:16.540041 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466" Jul 17 07:41:01.232770 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:01.232734 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466"] Jul 17 07:41:01.233201 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:01.232922 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466" podUID="772de92b-4481-406c-813d-908b0926d9eb" containerName="istio-proxy" containerID="cri-o://350675981c495504c3475292daf386ac3c5f4a374c6fe562ad95c7f69a101d95" gracePeriod=30 Jul 17 07:41:01.541465 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:01.541375 2568 patch_prober.go:28] interesting pod/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 17 07:41:01.541465 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:01.541444 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466" podUID="772de92b-4481-406c-813d-908b0926d9eb" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 17 07:41:06.484550 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:06.484524 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466" Jul 17 07:41:06.527886 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:06.527808 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7z622\" (UniqueName: \"kubernetes.io/projected/772de92b-4481-406c-813d-908b0926d9eb-kube-api-access-7z622\") pod \"772de92b-4481-406c-813d-908b0926d9eb\" (UID: \"772de92b-4481-406c-813d-908b0926d9eb\") " Jul 17 07:41:06.527886 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:06.527842 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/772de92b-4481-406c-813d-908b0926d9eb-istio-data\") pod \"772de92b-4481-406c-813d-908b0926d9eb\" (UID: \"772de92b-4481-406c-813d-908b0926d9eb\") " Jul 17 07:41:06.527886 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:06.527883 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/772de92b-4481-406c-813d-908b0926d9eb-istio-envoy\") pod \"772de92b-4481-406c-813d-908b0926d9eb\" (UID: \"772de92b-4481-406c-813d-908b0926d9eb\") " Jul 17 07:41:06.528159 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:06.527906 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/772de92b-4481-406c-813d-908b0926d9eb-istiod-ca-cert\") pod \"772de92b-4481-406c-813d-908b0926d9eb\" (UID: \"772de92b-4481-406c-813d-908b0926d9eb\") " Jul 17 07:41:06.528159 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:06.527960 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/772de92b-4481-406c-813d-908b0926d9eb-credential-socket\") pod \"772de92b-4481-406c-813d-908b0926d9eb\" (UID: \"772de92b-4481-406c-813d-908b0926d9eb\") " Jul 17 07:41:06.528159 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:06.527982 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/772de92b-4481-406c-813d-908b0926d9eb-workload-socket\") pod \"772de92b-4481-406c-813d-908b0926d9eb\" (UID: \"772de92b-4481-406c-813d-908b0926d9eb\") " Jul 17 07:41:06.528159 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:06.528009 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/772de92b-4481-406c-813d-908b0926d9eb-istio-token\") pod \"772de92b-4481-406c-813d-908b0926d9eb\" (UID: \"772de92b-4481-406c-813d-908b0926d9eb\") " Jul 17 07:41:06.528159 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:06.528095 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/772de92b-4481-406c-813d-908b0926d9eb-istio-podinfo\") pod \"772de92b-4481-406c-813d-908b0926d9eb\" (UID: \"772de92b-4481-406c-813d-908b0926d9eb\") " Jul 17 07:41:06.528159 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:06.528122 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/772de92b-4481-406c-813d-908b0926d9eb-workload-certs\") pod \"772de92b-4481-406c-813d-908b0926d9eb\" (UID: \"772de92b-4481-406c-813d-908b0926d9eb\") " Jul 17 07:41:06.528650 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:06.528623 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/772de92b-4481-406c-813d-908b0926d9eb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "772de92b-4481-406c-813d-908b0926d9eb" (UID: "772de92b-4481-406c-813d-908b0926d9eb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:41:06.529554 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:06.528996 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/772de92b-4481-406c-813d-908b0926d9eb-istio-data" (OuterVolumeSpecName: "istio-data") pod "772de92b-4481-406c-813d-908b0926d9eb" (UID: "772de92b-4481-406c-813d-908b0926d9eb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:41:06.529554 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:06.529148 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/772de92b-4481-406c-813d-908b0926d9eb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "772de92b-4481-406c-813d-908b0926d9eb" (UID: "772de92b-4481-406c-813d-908b0926d9eb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 07:41:06.529554 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:06.529451 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/772de92b-4481-406c-813d-908b0926d9eb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "772de92b-4481-406c-813d-908b0926d9eb" (UID: "772de92b-4481-406c-813d-908b0926d9eb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:41:06.529554 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:06.529518 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/772de92b-4481-406c-813d-908b0926d9eb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "772de92b-4481-406c-813d-908b0926d9eb" (UID: "772de92b-4481-406c-813d-908b0926d9eb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:41:06.530750 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:06.530671 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/772de92b-4481-406c-813d-908b0926d9eb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "772de92b-4481-406c-813d-908b0926d9eb" (UID: "772de92b-4481-406c-813d-908b0926d9eb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:41:06.531067 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:06.531039 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/772de92b-4481-406c-813d-908b0926d9eb-kube-api-access-7z622" (OuterVolumeSpecName: "kube-api-access-7z622") pod "772de92b-4481-406c-813d-908b0926d9eb" (UID: "772de92b-4481-406c-813d-908b0926d9eb"). InnerVolumeSpecName "kube-api-access-7z622". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 07:41:06.531970 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:06.531890 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/772de92b-4481-406c-813d-908b0926d9eb-istio-token" (OuterVolumeSpecName: "istio-token") pod "772de92b-4481-406c-813d-908b0926d9eb" (UID: "772de92b-4481-406c-813d-908b0926d9eb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 07:41:06.531970 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:06.531900 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/772de92b-4481-406c-813d-908b0926d9eb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "772de92b-4481-406c-813d-908b0926d9eb" (UID: "772de92b-4481-406c-813d-908b0926d9eb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 17 07:41:06.536331 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:06.536302 2568 generic.go:356] "Generic (PLEG): container finished" podID="772de92b-4481-406c-813d-908b0926d9eb" containerID="350675981c495504c3475292daf386ac3c5f4a374c6fe562ad95c7f69a101d95" exitCode=0 Jul 17 07:41:06.536443 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:06.536356 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466" Jul 17 07:41:06.536443 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:06.536376 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466" event={"ID":"772de92b-4481-406c-813d-908b0926d9eb","Type":"ContainerDied","Data":"350675981c495504c3475292daf386ac3c5f4a374c6fe562ad95c7f69a101d95"} Jul 17 07:41:06.536443 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:06.536431 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466" event={"ID":"772de92b-4481-406c-813d-908b0926d9eb","Type":"ContainerDied","Data":"4444b229e35f4b925ad8f3e381fce3170e87f9427b286e3a88c7d40d22caf759"} Jul 17 07:41:06.536625 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:06.536463 2568 scope.go:117] "RemoveContainer" containerID="350675981c495504c3475292daf386ac3c5f4a374c6fe562ad95c7f69a101d95" Jul 17 07:41:06.556930 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:06.556907 2568 scope.go:117] "RemoveContainer" containerID="98aea4dc4b5e8e21f96b239ae0560879c7fab4f4ff5d6c45ac5987d2385e4be9" Jul 17 07:41:06.566923 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:06.566902 2568 scope.go:117] "RemoveContainer" containerID="350675981c495504c3475292daf386ac3c5f4a374c6fe562ad95c7f69a101d95" Jul 17 07:41:06.567253 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:41:06.567233 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"350675981c495504c3475292daf386ac3c5f4a374c6fe562ad95c7f69a101d95\": container with ID starting with 350675981c495504c3475292daf386ac3c5f4a374c6fe562ad95c7f69a101d95 not found: ID does not exist" containerID="350675981c495504c3475292daf386ac3c5f4a374c6fe562ad95c7f69a101d95" Jul 17 07:41:06.567372 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:06.567267 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"350675981c495504c3475292daf386ac3c5f4a374c6fe562ad95c7f69a101d95"} err="failed to get container status \"350675981c495504c3475292daf386ac3c5f4a374c6fe562ad95c7f69a101d95\": rpc error: code = NotFound desc = could not find container \"350675981c495504c3475292daf386ac3c5f4a374c6fe562ad95c7f69a101d95\": container with ID starting with 350675981c495504c3475292daf386ac3c5f4a374c6fe562ad95c7f69a101d95 not found: ID does not exist" Jul 17 07:41:06.567372 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:06.567306 2568 scope.go:117] "RemoveContainer" containerID="98aea4dc4b5e8e21f96b239ae0560879c7fab4f4ff5d6c45ac5987d2385e4be9" Jul 17 07:41:06.567372 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:06.567359 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466"] Jul 17 07:41:06.567611 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:41:06.567586 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98aea4dc4b5e8e21f96b239ae0560879c7fab4f4ff5d6c45ac5987d2385e4be9\": container with ID starting with 98aea4dc4b5e8e21f96b239ae0560879c7fab4f4ff5d6c45ac5987d2385e4be9 not found: ID does not exist" containerID="98aea4dc4b5e8e21f96b239ae0560879c7fab4f4ff5d6c45ac5987d2385e4be9" Jul 17 07:41:06.567697 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:06.567620 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98aea4dc4b5e8e21f96b239ae0560879c7fab4f4ff5d6c45ac5987d2385e4be9"} err="failed to get container status \"98aea4dc4b5e8e21f96b239ae0560879c7fab4f4ff5d6c45ac5987d2385e4be9\": rpc error: code = NotFound desc = could not find container \"98aea4dc4b5e8e21f96b239ae0560879c7fab4f4ff5d6c45ac5987d2385e4be9\": container with ID starting with 98aea4dc4b5e8e21f96b239ae0560879c7fab4f4ff5d6c45ac5987d2385e4be9 not found: ID does not exist" Jul 17 07:41:06.572182 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:06.572161 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-b531d224-gw-openshift-default-799fddfbcf-wx466"] Jul 17 07:41:06.629703 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:06.629674 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7z622\" (UniqueName: \"kubernetes.io/projected/772de92b-4481-406c-813d-908b0926d9eb-kube-api-access-7z622\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:41:06.629703 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:06.629700 2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/772de92b-4481-406c-813d-908b0926d9eb-istio-data\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:41:06.629703 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:06.629709 2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/772de92b-4481-406c-813d-908b0926d9eb-istio-envoy\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:41:06.629929 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:06.629717 2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/772de92b-4481-406c-813d-908b0926d9eb-istiod-ca-cert\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:41:06.629929 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:06.629725 2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/772de92b-4481-406c-813d-908b0926d9eb-credential-socket\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:41:06.629929 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:06.629734 2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/772de92b-4481-406c-813d-908b0926d9eb-workload-socket\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:41:06.629929 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:06.629742 2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/772de92b-4481-406c-813d-908b0926d9eb-istio-token\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:41:06.629929 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:06.629751 2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/772de92b-4481-406c-813d-908b0926d9eb-istio-podinfo\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:41:06.629929 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:06.629758 2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/772de92b-4481-406c-813d-908b0926d9eb-workload-certs\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:41:08.000040 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:08.000007 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="772de92b-4481-406c-813d-908b0926d9eb" path="/var/lib/kubelet/pods/772de92b-4481-406c-813d-908b0926d9eb/volumes" Jul 17 07:41:08.581574 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:08.581543 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96"] Jul 17 07:41:08.581950 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:08.581936 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="772de92b-4481-406c-813d-908b0926d9eb" containerName="istio-proxy" Jul 17 07:41:08.582006 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:08.581951 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="772de92b-4481-406c-813d-908b0926d9eb" containerName="istio-proxy" Jul 17 07:41:08.582041 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:08.582019 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="772de92b-4481-406c-813d-908b0926d9eb" containerName="istio-proxy" Jul 17 07:41:08.585160 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:08.585143 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96" Jul 17 07:41:08.588008 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:08.587988 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-370a9d5a-openshift-default-dockercfg-9mljt\"" Jul 17 07:41:08.598476 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:08.598453 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96"] Jul 17 07:41:08.648221 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:08.648186 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1544029-0f8f-43fd-a078-b5bda4e622da-istio-envoy\") pod \"e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96\" (UID: \"c1544029-0f8f-43fd-a078-b5bda4e622da\") " pod="openshift-ingress/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96" Jul 17 07:41:08.648406 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:08.648225 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1544029-0f8f-43fd-a078-b5bda4e622da-workload-socket\") pod \"e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96\" (UID: \"c1544029-0f8f-43fd-a078-b5bda4e622da\") " pod="openshift-ingress/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96" Jul 17 07:41:08.648406 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:08.648260 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1544029-0f8f-43fd-a078-b5bda4e622da-istio-data\") pod \"e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96\" (UID: \"c1544029-0f8f-43fd-a078-b5bda4e622da\") " pod="openshift-ingress/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96" Jul 17 07:41:08.648406 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:08.648311 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqf9s\" (UniqueName: \"kubernetes.io/projected/c1544029-0f8f-43fd-a078-b5bda4e622da-kube-api-access-cqf9s\") pod \"e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96\" (UID: \"c1544029-0f8f-43fd-a078-b5bda4e622da\") " pod="openshift-ingress/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96" Jul 17 07:41:08.648406 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:08.648350 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1544029-0f8f-43fd-a078-b5bda4e622da-credential-socket\") pod \"e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96\" (UID: \"c1544029-0f8f-43fd-a078-b5bda4e622da\") " pod="openshift-ingress/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96" Jul 17 07:41:08.648406 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:08.648371 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1544029-0f8f-43fd-a078-b5bda4e622da-workload-certs\") pod \"e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96\" (UID: \"c1544029-0f8f-43fd-a078-b5bda4e622da\") " pod="openshift-ingress/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96" Jul 17 07:41:08.648649 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:08.648455 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1544029-0f8f-43fd-a078-b5bda4e622da-istio-podinfo\") pod \"e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96\" (UID: \"c1544029-0f8f-43fd-a078-b5bda4e622da\") " pod="openshift-ingress/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96" Jul 17 07:41:08.648649 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:08.648491 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1544029-0f8f-43fd-a078-b5bda4e622da-istio-token\") pod \"e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96\" (UID: \"c1544029-0f8f-43fd-a078-b5bda4e622da\") " pod="openshift-ingress/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96" Jul 17 07:41:08.648649 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:08.648547 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1544029-0f8f-43fd-a078-b5bda4e622da-istiod-ca-cert\") pod \"e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96\" (UID: \"c1544029-0f8f-43fd-a078-b5bda4e622da\") " pod="openshift-ingress/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96" Jul 17 07:41:08.749621 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:08.749579 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1544029-0f8f-43fd-a078-b5bda4e622da-istio-envoy\") pod \"e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96\" (UID: \"c1544029-0f8f-43fd-a078-b5bda4e622da\") " pod="openshift-ingress/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96" Jul 17 07:41:08.749621 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:08.749626 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1544029-0f8f-43fd-a078-b5bda4e622da-workload-socket\") pod \"e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96\" (UID: \"c1544029-0f8f-43fd-a078-b5bda4e622da\") " pod="openshift-ingress/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96" Jul 17 07:41:08.749892 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:08.749668 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1544029-0f8f-43fd-a078-b5bda4e622da-istio-data\") pod \"e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96\" (UID: \"c1544029-0f8f-43fd-a078-b5bda4e622da\") " pod="openshift-ingress/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96" Jul 17 07:41:08.749892 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:08.749695 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqf9s\" (UniqueName: \"kubernetes.io/projected/c1544029-0f8f-43fd-a078-b5bda4e622da-kube-api-access-cqf9s\") pod \"e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96\" (UID: \"c1544029-0f8f-43fd-a078-b5bda4e622da\") " pod="openshift-ingress/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96" Jul 17 07:41:08.749892 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:08.749734 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1544029-0f8f-43fd-a078-b5bda4e622da-credential-socket\") pod \"e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96\" (UID: \"c1544029-0f8f-43fd-a078-b5bda4e622da\") " pod="openshift-ingress/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96" Jul 17 07:41:08.749892 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:08.749761 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1544029-0f8f-43fd-a078-b5bda4e622da-workload-certs\") pod \"e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96\" (UID: \"c1544029-0f8f-43fd-a078-b5bda4e622da\") " pod="openshift-ingress/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96" Jul 17 07:41:08.749892 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:08.749798 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1544029-0f8f-43fd-a078-b5bda4e622da-istio-podinfo\") pod \"e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96\" (UID: \"c1544029-0f8f-43fd-a078-b5bda4e622da\") " pod="openshift-ingress/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96" Jul 17 07:41:08.749892 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:08.749831 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1544029-0f8f-43fd-a078-b5bda4e622da-istio-token\") pod \"e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96\" (UID: \"c1544029-0f8f-43fd-a078-b5bda4e622da\") " pod="openshift-ingress/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96" Jul 17 07:41:08.749892 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:08.749867 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1544029-0f8f-43fd-a078-b5bda4e622da-istiod-ca-cert\") pod \"e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96\" (UID: \"c1544029-0f8f-43fd-a078-b5bda4e622da\") " pod="openshift-ingress/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96" Jul 17 07:41:08.750246 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:08.750117 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1544029-0f8f-43fd-a078-b5bda4e622da-credential-socket\") pod \"e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96\" (UID: \"c1544029-0f8f-43fd-a078-b5bda4e622da\") " pod="openshift-ingress/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96" Jul 17 07:41:08.750246 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:08.750166 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1544029-0f8f-43fd-a078-b5bda4e622da-istio-data\") pod \"e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96\" (UID: \"c1544029-0f8f-43fd-a078-b5bda4e622da\") " pod="openshift-ingress/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96" Jul 17 07:41:08.750246 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:08.750202 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1544029-0f8f-43fd-a078-b5bda4e622da-workload-socket\") pod \"e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96\" (UID: \"c1544029-0f8f-43fd-a078-b5bda4e622da\") " pod="openshift-ingress/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96" Jul 17 07:41:08.750479 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:08.750456 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1544029-0f8f-43fd-a078-b5bda4e622da-workload-certs\") pod \"e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96\" (UID: \"c1544029-0f8f-43fd-a078-b5bda4e622da\") " pod="openshift-ingress/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96" Jul 17 07:41:08.750659 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:08.750640 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1544029-0f8f-43fd-a078-b5bda4e622da-istiod-ca-cert\") pod \"e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96\" (UID: \"c1544029-0f8f-43fd-a078-b5bda4e622da\") " pod="openshift-ingress/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96" Jul 17 07:41:08.751866 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:08.751839 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1544029-0f8f-43fd-a078-b5bda4e622da-istio-envoy\") pod \"e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96\" (UID: \"c1544029-0f8f-43fd-a078-b5bda4e622da\") " pod="openshift-ingress/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96" Jul 17 07:41:08.752250 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:08.752232 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1544029-0f8f-43fd-a078-b5bda4e622da-istio-podinfo\") pod \"e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96\" (UID: \"c1544029-0f8f-43fd-a078-b5bda4e622da\") " pod="openshift-ingress/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96" Jul 17 07:41:08.758968 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:08.758948 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1544029-0f8f-43fd-a078-b5bda4e622da-istio-token\") pod \"e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96\" (UID: \"c1544029-0f8f-43fd-a078-b5bda4e622da\") " pod="openshift-ingress/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96" Jul 17 07:41:08.759212 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:08.759196 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqf9s\" (UniqueName: \"kubernetes.io/projected/c1544029-0f8f-43fd-a078-b5bda4e622da-kube-api-access-cqf9s\") pod \"e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96\" (UID: \"c1544029-0f8f-43fd-a078-b5bda4e622da\") " pod="openshift-ingress/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96" Jul 17 07:41:08.896667 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:08.896641 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96" Jul 17 07:41:09.040792 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:09.040761 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96"] Jul 17 07:41:09.042140 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:41:09.042116 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1544029_0f8f_43fd_a078_b5bda4e622da.slice/crio-71bb26dc8708667eb6ae69cbfb9ec932e14360acccd8cee2db574dc1d5d22ab3 WatchSource:0}: Error finding container 71bb26dc8708667eb6ae69cbfb9ec932e14360acccd8cee2db574dc1d5d22ab3: Status 404 returned error can't find the container with id 71bb26dc8708667eb6ae69cbfb9ec932e14360acccd8cee2db574dc1d5d22ab3 Jul 17 07:41:09.044213 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:09.044177 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:41:09.044326 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:09.044245 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:41:09.044326 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:09.044270 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:41:09.555142 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:09.555106 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96" event={"ID":"c1544029-0f8f-43fd-a078-b5bda4e622da","Type":"ContainerStarted","Data":"07344e0b72fd927e9f803d47c74250581365c03fcb7b51da8df75242c588451c"} Jul 17 07:41:09.555383 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:09.555164 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96" event={"ID":"c1544029-0f8f-43fd-a078-b5bda4e622da","Type":"ContainerStarted","Data":"71bb26dc8708667eb6ae69cbfb9ec932e14360acccd8cee2db574dc1d5d22ab3"} Jul 17 07:41:09.578565 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:09.578514 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96" podStartSLOduration=1.578498717 podStartE2EDuration="1.578498717s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 07:41:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:41:09.576637843 +0000 UTC m=+1774.061326817" watchObservedRunningTime="2026-07-17 07:41:09.578498717 +0000 UTC m=+1774.063187691" Jul 17 07:41:09.897189 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:09.897149 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96" Jul 17 07:41:10.901598 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:10.901563 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96" Jul 17 07:41:11.567038 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:11.566998 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96" Jul 17 07:41:11.568020 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:11.567996 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96" Jul 17 07:41:14.887712 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:14.887668 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-370a9d5a-56zb6"] Jul 17 07:41:14.888191 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:14.888076 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="772de92b-4481-406c-813d-908b0926d9eb" containerName="istio-proxy" Jul 17 07:41:14.888191 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:14.888087 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="772de92b-4481-406c-813d-908b0926d9eb" containerName="istio-proxy" Jul 17 07:41:14.888191 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:14.888168 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="772de92b-4481-406c-813d-908b0926d9eb" containerName="istio-proxy" Jul 17 07:41:14.891491 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:14.891467 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-370a9d5a-56zb6" Jul 17 07:41:14.895708 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:14.895689 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-c65vn\"" Jul 17 07:41:14.906571 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:14.906549 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-370a9d5a-56zb6"] Jul 17 07:41:15.011776 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:15.011741 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85a5bcef-6272-40c2-9bb9-836ee75525f2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-370a9d5a-56zb6\" (UID: \"85a5bcef-6272-40c2-9bb9-836ee75525f2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-370a9d5a-56zb6" Jul 17 07:41:15.112331 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:15.112271 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85a5bcef-6272-40c2-9bb9-836ee75525f2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-370a9d5a-56zb6\" (UID: \"85a5bcef-6272-40c2-9bb9-836ee75525f2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-370a9d5a-56zb6" Jul 17 07:41:15.112879 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:15.112856 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85a5bcef-6272-40c2-9bb9-836ee75525f2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-370a9d5a-56zb6\" (UID: \"85a5bcef-6272-40c2-9bb9-836ee75525f2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-370a9d5a-56zb6" Jul 17 07:41:15.201787 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:15.201707 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-370a9d5a-56zb6" Jul 17 07:41:15.381966 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:15.381939 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-370a9d5a-56zb6"] Jul 17 07:41:15.382907 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:41:15.382884 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85a5bcef_6272_40c2_9bb9_836ee75525f2.slice/crio-569ced9bd81906639952dc40a4d2d90330109f1d0c7b2b4c5c9cff915bb68674 WatchSource:0}: Error finding container 569ced9bd81906639952dc40a4d2d90330109f1d0c7b2b4c5c9cff915bb68674: Status 404 returned error can't find the container with id 569ced9bd81906639952dc40a4d2d90330109f1d0c7b2b4c5c9cff915bb68674 Jul 17 07:41:15.384827 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:15.384810 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 17 07:41:15.589756 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:15.589670 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-370a9d5a-56zb6" event={"ID":"85a5bcef-6272-40c2-9bb9-836ee75525f2","Type":"ContainerStarted","Data":"569ced9bd81906639952dc40a4d2d90330109f1d0c7b2b4c5c9cff915bb68674"} Jul 17 07:41:23.643018 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:23.642978 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-370a9d5a-56zb6" event={"ID":"85a5bcef-6272-40c2-9bb9-836ee75525f2","Type":"ContainerStarted","Data":"99117f8bd53501bc430716303203b8e72c71fdd0e5c27dd1422cc5907f146749"} Jul 17 07:41:23.661050 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:23.660994 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-370a9d5a-56zb6" podStartSLOduration=2.016114932 podStartE2EDuration="9.660978604s" totalImagesPullingTime="7.644863672s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 07:41:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:41:23.658563093 +0000 UTC m=+1788.143252067" watchObservedRunningTime="2026-07-17 07:41:23.660978604 +0000 UTC m=+1788.145667579" Jul 17 07:41:24.650393 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:24.650358 2568 generic.go:356] "Generic (PLEG): container finished" podID="85a5bcef-6272-40c2-9bb9-836ee75525f2" containerID="99117f8bd53501bc430716303203b8e72c71fdd0e5c27dd1422cc5907f146749" exitCode=7 Jul 17 07:41:24.650925 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:24.650457 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-370a9d5a-56zb6" event={"ID":"85a5bcef-6272-40c2-9bb9-836ee75525f2","Type":"ContainerDied","Data":"99117f8bd53501bc430716303203b8e72c71fdd0e5c27dd1422cc5907f146749"} Jul 17 07:41:24.650925 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:24.650656 2568 scope.go:117] "RemoveContainer" containerID="99117f8bd53501bc430716303203b8e72c71fdd0e5c27dd1422cc5907f146749" Jul 17 07:41:25.657637 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:25.657542 2568 generic.go:356] "Generic (PLEG): container finished" podID="85a5bcef-6272-40c2-9bb9-836ee75525f2" containerID="c269380d26c5bec3d5192c798b29d19cd3db88f85b66fb651f8c781e12651942" exitCode=7 Jul 17 07:41:25.657637 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:25.657616 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-370a9d5a-56zb6" event={"ID":"85a5bcef-6272-40c2-9bb9-836ee75525f2","Type":"ContainerDied","Data":"c269380d26c5bec3d5192c798b29d19cd3db88f85b66fb651f8c781e12651942"} Jul 17 07:41:25.658134 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:25.657677 2568 scope.go:117] "RemoveContainer" containerID="99117f8bd53501bc430716303203b8e72c71fdd0e5c27dd1422cc5907f146749" Jul 17 07:41:25.658134 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:25.657943 2568 scope.go:117] "RemoveContainer" containerID="c269380d26c5bec3d5192c798b29d19cd3db88f85b66fb651f8c781e12651942" Jul 17 07:41:25.658238 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:41:25.658217 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-370a9d5a-56zb6_odh-ai-gateway-infra(85a5bcef-6272-40c2-9bb9-836ee75525f2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-370a9d5a-56zb6" podUID="85a5bcef-6272-40c2-9bb9-836ee75525f2" Jul 17 07:41:25.659422 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:25.659401 2568 generic.go:356] "Generic (PLEG): container finished" podID="c1544029-0f8f-43fd-a078-b5bda4e622da" containerID="07344e0b72fd927e9f803d47c74250581365c03fcb7b51da8df75242c588451c" exitCode=0 Jul 17 07:41:25.659519 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:25.659475 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96" event={"ID":"c1544029-0f8f-43fd-a078-b5bda4e622da","Type":"ContainerDied","Data":"07344e0b72fd927e9f803d47c74250581365c03fcb7b51da8df75242c588451c"} Jul 17 07:41:25.659811 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:25.659798 2568 scope.go:117] "RemoveContainer" containerID="07344e0b72fd927e9f803d47c74250581365c03fcb7b51da8df75242c588451c" Jul 17 07:41:25.660746 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:25.660716 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:41:25.660857 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:25.660781 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:41:25.660857 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:25.660812 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:41:26.567865 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:26.567814 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96" Jul 17 07:41:26.667577 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:26.666809 2568 scope.go:117] "RemoveContainer" containerID="c269380d26c5bec3d5192c798b29d19cd3db88f85b66fb651f8c781e12651942" Jul 17 07:41:26.667577 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:41:26.667078 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-370a9d5a-56zb6_odh-ai-gateway-infra(85a5bcef-6272-40c2-9bb9-836ee75525f2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-370a9d5a-56zb6" podUID="85a5bcef-6272-40c2-9bb9-836ee75525f2" Jul 17 07:41:26.673142 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:26.673107 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96" event={"ID":"c1544029-0f8f-43fd-a078-b5bda4e622da","Type":"ContainerStarted","Data":"c2ea136eb457fb0088081fcff4f2c8e62c511d5fc3720869a4b55472341a0785"} Jul 17 07:41:26.897207 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:26.897166 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96" Jul 17 07:41:26.898491 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:26.898462 2568 patch_prober.go:28] interesting pod/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 17 07:41:26.898622 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:26.898527 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96" podUID="c1544029-0f8f-43fd-a078-b5bda4e622da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 17 07:41:27.897732 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:27.897692 2568 patch_prober.go:28] interesting pod/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 17 07:41:27.898140 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:27.897762 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96" podUID="c1544029-0f8f-43fd-a078-b5bda4e622da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 17 07:41:28.897940 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:28.897899 2568 patch_prober.go:28] interesting pod/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 17 07:41:28.898412 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:28.897959 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96" podUID="c1544029-0f8f-43fd-a078-b5bda4e622da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 17 07:41:29.897250 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:29.897209 2568 patch_prober.go:28] interesting pod/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 17 07:41:29.897502 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:29.897329 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96" podUID="c1544029-0f8f-43fd-a078-b5bda4e622da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 17 07:41:30.901659 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:30.901629 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96" Jul 17 07:41:30.902141 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:30.901974 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96" Jul 17 07:41:30.902877 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:30.902849 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96" Jul 17 07:41:36.136160 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:36.136127 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-xr6pg_2ce00a54-15e9-42b7-b3a8-9e58d1101558/cluster-monitoring-operator/0.log" Jul 17 07:41:36.140823 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:36.140801 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-xr6pg_2ce00a54-15e9-42b7-b3a8-9e58d1101558/cluster-monitoring-operator/0.log" Jul 17 07:41:36.141615 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:36.141590 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-xpxv5_27538d1e-992b-4651-a2d2-7a7ecf442806/console-operator/1.log" Jul 17 07:41:36.145462 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:36.145442 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cpwz4_1c44a3ae-4c83-4203-8b0b-116d6618accf/ovn-acl-logging/0.log" Jul 17 07:41:36.146175 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:36.146142 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-xpxv5_27538d1e-992b-4651-a2d2-7a7ecf442806/console-operator/1.log" Jul 17 07:41:36.149716 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:36.149695 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cpwz4_1c44a3ae-4c83-4203-8b0b-116d6618accf/ovn-acl-logging/0.log" Jul 17 07:41:36.994241 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:36.994212 2568 scope.go:117] "RemoveContainer" containerID="c269380d26c5bec3d5192c798b29d19cd3db88f85b66fb651f8c781e12651942" Jul 17 07:41:37.743713 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:37.743677 2568 generic.go:356] "Generic (PLEG): container finished" podID="85a5bcef-6272-40c2-9bb9-836ee75525f2" containerID="f61c2358704e0d0e8a91098dbe2121758d3e409ae786a6e2c58d6eba984b84c8" exitCode=7 Jul 17 07:41:37.744113 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:37.743750 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-370a9d5a-56zb6" event={"ID":"85a5bcef-6272-40c2-9bb9-836ee75525f2","Type":"ContainerDied","Data":"f61c2358704e0d0e8a91098dbe2121758d3e409ae786a6e2c58d6eba984b84c8"} Jul 17 07:41:37.744113 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:37.743805 2568 scope.go:117] "RemoveContainer" containerID="c269380d26c5bec3d5192c798b29d19cd3db88f85b66fb651f8c781e12651942" Jul 17 07:41:37.744202 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:37.744128 2568 scope.go:117] "RemoveContainer" containerID="f61c2358704e0d0e8a91098dbe2121758d3e409ae786a6e2c58d6eba984b84c8" Jul 17 07:41:37.744391 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:41:37.744372 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-370a9d5a-56zb6_odh-ai-gateway-infra(85a5bcef-6272-40c2-9bb9-836ee75525f2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-370a9d5a-56zb6" podUID="85a5bcef-6272-40c2-9bb9-836ee75525f2" Jul 17 07:41:38.785720 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:38.785686 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-370a9d5a-56zb6"] Jul 17 07:41:38.928639 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:38.928613 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-370a9d5a-56zb6" Jul 17 07:41:39.047125 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:39.047048 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85a5bcef-6272-40c2-9bb9-836ee75525f2-maas-api-service-ca\") pod \"85a5bcef-6272-40c2-9bb9-836ee75525f2\" (UID: \"85a5bcef-6272-40c2-9bb9-836ee75525f2\") " Jul 17 07:41:39.047420 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:39.047398 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85a5bcef-6272-40c2-9bb9-836ee75525f2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "85a5bcef-6272-40c2-9bb9-836ee75525f2" (UID: "85a5bcef-6272-40c2-9bb9-836ee75525f2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 07:41:39.148076 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:39.148048 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85a5bcef-6272-40c2-9bb9-836ee75525f2-maas-api-service-ca\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:41:39.757847 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:39.757819 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-370a9d5a-56zb6" Jul 17 07:41:39.758021 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:39.757861 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-370a9d5a-56zb6" event={"ID":"85a5bcef-6272-40c2-9bb9-836ee75525f2","Type":"ContainerDied","Data":"569ced9bd81906639952dc40a4d2d90330109f1d0c7b2b4c5c9cff915bb68674"} Jul 17 07:41:39.758021 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:39.757950 2568 scope.go:117] "RemoveContainer" containerID="f61c2358704e0d0e8a91098dbe2121758d3e409ae786a6e2c58d6eba984b84c8" Jul 17 07:41:39.800205 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:39.800176 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-370a9d5a-56zb6"] Jul 17 07:41:39.821550 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:39.821522 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-370a9d5a-56zb6"] Jul 17 07:41:40.000038 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:41:40.000002 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85a5bcef-6272-40c2-9bb9-836ee75525f2" path="/var/lib/kubelet/pods/85a5bcef-6272-40c2-9bb9-836ee75525f2/volumes" Jul 17 07:46:36.174217 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:46:36.174191 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-xr6pg_2ce00a54-15e9-42b7-b3a8-9e58d1101558/cluster-monitoring-operator/0.log" Jul 17 07:46:36.179156 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:46:36.179134 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-xpxv5_27538d1e-992b-4651-a2d2-7a7ecf442806/console-operator/1.log" Jul 17 07:46:36.182527 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:46:36.182503 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cpwz4_1c44a3ae-4c83-4203-8b0b-116d6618accf/ovn-acl-logging/0.log" Jul 17 07:46:36.182891 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:46:36.182864 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-xr6pg_2ce00a54-15e9-42b7-b3a8-9e58d1101558/cluster-monitoring-operator/0.log" Jul 17 07:46:36.187833 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:46:36.187816 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-xpxv5_27538d1e-992b-4651-a2d2-7a7ecf442806/console-operator/1.log" Jul 17 07:46:36.190912 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:46:36.190895 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cpwz4_1c44a3ae-4c83-4203-8b0b-116d6618accf/ovn-acl-logging/0.log" Jul 17 07:47:15.130597 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:15.130564 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96"] Jul 17 07:47:15.131152 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:15.130769 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96" podUID="c1544029-0f8f-43fd-a078-b5bda4e622da" containerName="istio-proxy" containerID="cri-o://c2ea136eb457fb0088081fcff4f2c8e62c511d5fc3720869a4b55472341a0785" gracePeriod=30 Jul 17 07:47:15.904452 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:15.904415 2568 patch_prober.go:28] interesting pod/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 17 07:47:15.904610 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:15.904475 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96" podUID="c1544029-0f8f-43fd-a078-b5bda4e622da" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 17 07:47:20.388169 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:20.388139 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96" Jul 17 07:47:20.541075 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:20.541044 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1544029-0f8f-43fd-a078-b5bda4e622da-workload-socket\") pod \"c1544029-0f8f-43fd-a078-b5bda4e622da\" (UID: \"c1544029-0f8f-43fd-a078-b5bda4e622da\") " Jul 17 07:47:20.541075 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:20.541075 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1544029-0f8f-43fd-a078-b5bda4e622da-istio-data\") pod \"c1544029-0f8f-43fd-a078-b5bda4e622da\" (UID: \"c1544029-0f8f-43fd-a078-b5bda4e622da\") " Jul 17 07:47:20.541319 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:20.541102 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1544029-0f8f-43fd-a078-b5bda4e622da-credential-socket\") pod \"c1544029-0f8f-43fd-a078-b5bda4e622da\" (UID: \"c1544029-0f8f-43fd-a078-b5bda4e622da\") " Jul 17 07:47:20.541319 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:20.541139 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1544029-0f8f-43fd-a078-b5bda4e622da-workload-certs\") pod \"c1544029-0f8f-43fd-a078-b5bda4e622da\" (UID: \"c1544029-0f8f-43fd-a078-b5bda4e622da\") " Jul 17 07:47:20.541319 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:20.541156 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1544029-0f8f-43fd-a078-b5bda4e622da-istio-podinfo\") pod \"c1544029-0f8f-43fd-a078-b5bda4e622da\" (UID: \"c1544029-0f8f-43fd-a078-b5bda4e622da\") " Jul 17 07:47:20.541319 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:20.541181 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cqf9s\" (UniqueName: \"kubernetes.io/projected/c1544029-0f8f-43fd-a078-b5bda4e622da-kube-api-access-cqf9s\") pod \"c1544029-0f8f-43fd-a078-b5bda4e622da\" (UID: \"c1544029-0f8f-43fd-a078-b5bda4e622da\") " Jul 17 07:47:20.541319 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:20.541199 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1544029-0f8f-43fd-a078-b5bda4e622da-istio-token\") pod \"c1544029-0f8f-43fd-a078-b5bda4e622da\" (UID: \"c1544029-0f8f-43fd-a078-b5bda4e622da\") " Jul 17 07:47:20.541319 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:20.541218 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1544029-0f8f-43fd-a078-b5bda4e622da-istio-envoy\") pod \"c1544029-0f8f-43fd-a078-b5bda4e622da\" (UID: \"c1544029-0f8f-43fd-a078-b5bda4e622da\") " Jul 17 07:47:20.541655 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:20.541326 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1544029-0f8f-43fd-a078-b5bda4e622da-istiod-ca-cert\") pod \"c1544029-0f8f-43fd-a078-b5bda4e622da\" (UID: \"c1544029-0f8f-43fd-a078-b5bda4e622da\") " Jul 17 07:47:20.541655 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:20.541381 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1544029-0f8f-43fd-a078-b5bda4e622da-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c1544029-0f8f-43fd-a078-b5bda4e622da" (UID: "c1544029-0f8f-43fd-a078-b5bda4e622da"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:47:20.541655 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:20.541495 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1544029-0f8f-43fd-a078-b5bda4e622da-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c1544029-0f8f-43fd-a078-b5bda4e622da" (UID: "c1544029-0f8f-43fd-a078-b5bda4e622da"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:47:20.541835 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:20.541677 2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1544029-0f8f-43fd-a078-b5bda4e622da-workload-certs\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:47:20.541835 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:20.541696 2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1544029-0f8f-43fd-a078-b5bda4e622da-workload-socket\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:47:20.541835 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:20.541738 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1544029-0f8f-43fd-a078-b5bda4e622da-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c1544029-0f8f-43fd-a078-b5bda4e622da" (UID: "c1544029-0f8f-43fd-a078-b5bda4e622da"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:47:20.541835 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:20.541791 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1544029-0f8f-43fd-a078-b5bda4e622da-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c1544029-0f8f-43fd-a078-b5bda4e622da" (UID: "c1544029-0f8f-43fd-a078-b5bda4e622da"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 07:47:20.542052 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:20.541928 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1544029-0f8f-43fd-a078-b5bda4e622da-istio-data" (OuterVolumeSpecName: "istio-data") pod "c1544029-0f8f-43fd-a078-b5bda4e622da" (UID: "c1544029-0f8f-43fd-a078-b5bda4e622da"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:47:20.543385 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:20.543363 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1544029-0f8f-43fd-a078-b5bda4e622da-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c1544029-0f8f-43fd-a078-b5bda4e622da" (UID: "c1544029-0f8f-43fd-a078-b5bda4e622da"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:47:20.543511 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:20.543460 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1544029-0f8f-43fd-a078-b5bda4e622da-kube-api-access-cqf9s" (OuterVolumeSpecName: "kube-api-access-cqf9s") pod "c1544029-0f8f-43fd-a078-b5bda4e622da" (UID: "c1544029-0f8f-43fd-a078-b5bda4e622da"). InnerVolumeSpecName "kube-api-access-cqf9s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 07:47:20.543672 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:20.543644 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c1544029-0f8f-43fd-a078-b5bda4e622da-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c1544029-0f8f-43fd-a078-b5bda4e622da" (UID: "c1544029-0f8f-43fd-a078-b5bda4e622da"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 17 07:47:20.543766 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:20.543735 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1544029-0f8f-43fd-a078-b5bda4e622da-istio-token" (OuterVolumeSpecName: "istio-token") pod "c1544029-0f8f-43fd-a078-b5bda4e622da" (UID: "c1544029-0f8f-43fd-a078-b5bda4e622da"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 07:47:20.632428 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:20.632396 2568 generic.go:356] "Generic (PLEG): container finished" podID="c1544029-0f8f-43fd-a078-b5bda4e622da" containerID="c2ea136eb457fb0088081fcff4f2c8e62c511d5fc3720869a4b55472341a0785" exitCode=0 Jul 17 07:47:20.632667 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:20.632451 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96" Jul 17 07:47:20.632667 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:20.632461 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96" event={"ID":"c1544029-0f8f-43fd-a078-b5bda4e622da","Type":"ContainerDied","Data":"c2ea136eb457fb0088081fcff4f2c8e62c511d5fc3720869a4b55472341a0785"} Jul 17 07:47:20.632667 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:20.632504 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96" event={"ID":"c1544029-0f8f-43fd-a078-b5bda4e622da","Type":"ContainerDied","Data":"71bb26dc8708667eb6ae69cbfb9ec932e14360acccd8cee2db574dc1d5d22ab3"} Jul 17 07:47:20.632667 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:20.632526 2568 scope.go:117] "RemoveContainer" containerID="c2ea136eb457fb0088081fcff4f2c8e62c511d5fc3720869a4b55472341a0785" Jul 17 07:47:20.642551 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:20.642532 2568 scope.go:117] "RemoveContainer" containerID="07344e0b72fd927e9f803d47c74250581365c03fcb7b51da8df75242c588451c" Jul 17 07:47:20.642749 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:20.642731 2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1544029-0f8f-43fd-a078-b5bda4e622da-istio-data\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:47:20.642836 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:20.642757 2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1544029-0f8f-43fd-a078-b5bda4e622da-credential-socket\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:47:20.642836 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:20.642771 2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1544029-0f8f-43fd-a078-b5bda4e622da-istio-podinfo\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:47:20.642836 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:20.642785 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cqf9s\" (UniqueName: \"kubernetes.io/projected/c1544029-0f8f-43fd-a078-b5bda4e622da-kube-api-access-cqf9s\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:47:20.642836 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:20.642798 2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1544029-0f8f-43fd-a078-b5bda4e622da-istio-token\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:47:20.642836 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:20.642813 2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1544029-0f8f-43fd-a078-b5bda4e622da-istio-envoy\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:47:20.642836 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:20.642826 2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1544029-0f8f-43fd-a078-b5bda4e622da-istiod-ca-cert\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:47:20.650617 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:20.650599 2568 scope.go:117] "RemoveContainer" containerID="c2ea136eb457fb0088081fcff4f2c8e62c511d5fc3720869a4b55472341a0785" Jul 17 07:47:20.650872 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:47:20.650856 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2ea136eb457fb0088081fcff4f2c8e62c511d5fc3720869a4b55472341a0785\": container with ID starting with c2ea136eb457fb0088081fcff4f2c8e62c511d5fc3720869a4b55472341a0785 not found: ID does not exist" containerID="c2ea136eb457fb0088081fcff4f2c8e62c511d5fc3720869a4b55472341a0785" Jul 17 07:47:20.650930 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:20.650878 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2ea136eb457fb0088081fcff4f2c8e62c511d5fc3720869a4b55472341a0785"} err="failed to get container status \"c2ea136eb457fb0088081fcff4f2c8e62c511d5fc3720869a4b55472341a0785\": rpc error: code = NotFound desc = could not find container \"c2ea136eb457fb0088081fcff4f2c8e62c511d5fc3720869a4b55472341a0785\": container with ID starting with c2ea136eb457fb0088081fcff4f2c8e62c511d5fc3720869a4b55472341a0785 not found: ID does not exist" Jul 17 07:47:20.650930 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:20.650893 2568 scope.go:117] "RemoveContainer" containerID="07344e0b72fd927e9f803d47c74250581365c03fcb7b51da8df75242c588451c" Jul 17 07:47:20.651100 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:47:20.651083 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07344e0b72fd927e9f803d47c74250581365c03fcb7b51da8df75242c588451c\": container with ID starting with 07344e0b72fd927e9f803d47c74250581365c03fcb7b51da8df75242c588451c not found: ID does not exist" containerID="07344e0b72fd927e9f803d47c74250581365c03fcb7b51da8df75242c588451c" Jul 17 07:47:20.651142 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:20.651103 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07344e0b72fd927e9f803d47c74250581365c03fcb7b51da8df75242c588451c"} err="failed to get container status \"07344e0b72fd927e9f803d47c74250581365c03fcb7b51da8df75242c588451c\": rpc error: code = NotFound desc = could not find container \"07344e0b72fd927e9f803d47c74250581365c03fcb7b51da8df75242c588451c\": container with ID starting with 07344e0b72fd927e9f803d47c74250581365c03fcb7b51da8df75242c588451c not found: ID does not exist" Jul 17 07:47:20.663667 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:20.663640 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96"] Jul 17 07:47:20.665864 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:20.665844 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-370a9d5a-openshift-default-64bf98fbc-5nl96"] Jul 17 07:47:21.999557 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:21.999514 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1544029-0f8f-43fd-a078-b5bda4e622da" path="/var/lib/kubelet/pods/c1544029-0f8f-43fd-a078-b5bda4e622da/volumes" Jul 17 07:47:27.659740 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:27.659704 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq"] Jul 17 07:47:27.660334 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:27.660315 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85a5bcef-6272-40c2-9bb9-836ee75525f2" containerName="revoke-keys" Jul 17 07:47:27.660407 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:27.660339 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="85a5bcef-6272-40c2-9bb9-836ee75525f2" containerName="revoke-keys" Jul 17 07:47:27.660407 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:27.660370 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1544029-0f8f-43fd-a078-b5bda4e622da" containerName="istio-proxy" Jul 17 07:47:27.660407 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:27.660379 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1544029-0f8f-43fd-a078-b5bda4e622da" containerName="istio-proxy" Jul 17 07:47:27.660407 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:27.660397 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1544029-0f8f-43fd-a078-b5bda4e622da" containerName="istio-proxy" Jul 17 07:47:27.660407 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:27.660406 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1544029-0f8f-43fd-a078-b5bda4e622da" containerName="istio-proxy" Jul 17 07:47:27.660576 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:27.660426 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85a5bcef-6272-40c2-9bb9-836ee75525f2" containerName="revoke-keys" Jul 17 07:47:27.660576 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:27.660434 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="85a5bcef-6272-40c2-9bb9-836ee75525f2" containerName="revoke-keys" Jul 17 07:47:27.660576 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:27.660445 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85a5bcef-6272-40c2-9bb9-836ee75525f2" containerName="revoke-keys" Jul 17 07:47:27.660576 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:27.660453 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="85a5bcef-6272-40c2-9bb9-836ee75525f2" containerName="revoke-keys" Jul 17 07:47:27.660576 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:27.660552 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="85a5bcef-6272-40c2-9bb9-836ee75525f2" containerName="revoke-keys" Jul 17 07:47:27.660576 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:27.660565 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1544029-0f8f-43fd-a078-b5bda4e622da" containerName="istio-proxy" Jul 17 07:47:27.660757 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:27.660577 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1544029-0f8f-43fd-a078-b5bda4e622da" containerName="istio-proxy" Jul 17 07:47:27.660757 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:27.660589 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="85a5bcef-6272-40c2-9bb9-836ee75525f2" containerName="revoke-keys" Jul 17 07:47:27.665483 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:27.665467 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq" Jul 17 07:47:27.668908 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:27.668875 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-f44a26e0-openshift-default-dockercfg-pzn5f\"" Jul 17 07:47:27.681788 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:27.681761 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq"] Jul 17 07:47:27.799754 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:27.799710 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56a77e36-fa16-490b-b5bb-9172f69b62c8-istiod-ca-cert\") pod \"e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq\" (UID: \"56a77e36-fa16-490b-b5bb-9172f69b62c8\") " pod="openshift-ingress/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq" Jul 17 07:47:27.799754 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:27.799758 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56a77e36-fa16-490b-b5bb-9172f69b62c8-workload-socket\") pod \"e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq\" (UID: \"56a77e36-fa16-490b-b5bb-9172f69b62c8\") " pod="openshift-ingress/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq" Jul 17 07:47:27.799972 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:27.799865 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56a77e36-fa16-490b-b5bb-9172f69b62c8-istio-data\") pod \"e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq\" (UID: \"56a77e36-fa16-490b-b5bb-9172f69b62c8\") " pod="openshift-ingress/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq" Jul 17 07:47:27.799972 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:27.799900 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgj45\" (UniqueName: \"kubernetes.io/projected/56a77e36-fa16-490b-b5bb-9172f69b62c8-kube-api-access-xgj45\") pod \"e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq\" (UID: \"56a77e36-fa16-490b-b5bb-9172f69b62c8\") " pod="openshift-ingress/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq" Jul 17 07:47:27.799972 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:27.799930 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56a77e36-fa16-490b-b5bb-9172f69b62c8-istio-envoy\") pod \"e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq\" (UID: \"56a77e36-fa16-490b-b5bb-9172f69b62c8\") " pod="openshift-ingress/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq" Jul 17 07:47:27.799972 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:27.799946 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56a77e36-fa16-490b-b5bb-9172f69b62c8-istio-token\") pod \"e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq\" (UID: \"56a77e36-fa16-490b-b5bb-9172f69b62c8\") " pod="openshift-ingress/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq" Jul 17 07:47:27.800189 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:27.800021 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56a77e36-fa16-490b-b5bb-9172f69b62c8-credential-socket\") pod \"e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq\" (UID: \"56a77e36-fa16-490b-b5bb-9172f69b62c8\") " pod="openshift-ingress/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq" Jul 17 07:47:27.800189 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:27.800050 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56a77e36-fa16-490b-b5bb-9172f69b62c8-workload-certs\") pod \"e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq\" (UID: \"56a77e36-fa16-490b-b5bb-9172f69b62c8\") " pod="openshift-ingress/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq" Jul 17 07:47:27.800189 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:27.800122 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56a77e36-fa16-490b-b5bb-9172f69b62c8-istio-podinfo\") pod \"e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq\" (UID: \"56a77e36-fa16-490b-b5bb-9172f69b62c8\") " pod="openshift-ingress/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq" Jul 17 07:47:27.901310 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:27.901256 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56a77e36-fa16-490b-b5bb-9172f69b62c8-istiod-ca-cert\") pod \"e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq\" (UID: \"56a77e36-fa16-490b-b5bb-9172f69b62c8\") " pod="openshift-ingress/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq" Jul 17 07:47:27.901497 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:27.901318 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56a77e36-fa16-490b-b5bb-9172f69b62c8-workload-socket\") pod \"e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq\" (UID: \"56a77e36-fa16-490b-b5bb-9172f69b62c8\") " pod="openshift-ingress/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq" Jul 17 07:47:27.901497 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:27.901347 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56a77e36-fa16-490b-b5bb-9172f69b62c8-istio-data\") pod \"e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq\" (UID: \"56a77e36-fa16-490b-b5bb-9172f69b62c8\") " pod="openshift-ingress/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq" Jul 17 07:47:27.901497 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:27.901369 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgj45\" (UniqueName: \"kubernetes.io/projected/56a77e36-fa16-490b-b5bb-9172f69b62c8-kube-api-access-xgj45\") pod \"e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq\" (UID: \"56a77e36-fa16-490b-b5bb-9172f69b62c8\") " pod="openshift-ingress/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq" Jul 17 07:47:27.901497 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:27.901391 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56a77e36-fa16-490b-b5bb-9172f69b62c8-istio-envoy\") pod \"e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq\" (UID: \"56a77e36-fa16-490b-b5bb-9172f69b62c8\") " pod="openshift-ingress/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq" Jul 17 07:47:27.901497 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:27.901406 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56a77e36-fa16-490b-b5bb-9172f69b62c8-istio-token\") pod \"e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq\" (UID: \"56a77e36-fa16-490b-b5bb-9172f69b62c8\") " pod="openshift-ingress/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq" Jul 17 07:47:27.901497 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:27.901492 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56a77e36-fa16-490b-b5bb-9172f69b62c8-credential-socket\") pod \"e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq\" (UID: \"56a77e36-fa16-490b-b5bb-9172f69b62c8\") " pod="openshift-ingress/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq" Jul 17 07:47:27.901847 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:27.901531 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56a77e36-fa16-490b-b5bb-9172f69b62c8-workload-certs\") pod \"e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq\" (UID: \"56a77e36-fa16-490b-b5bb-9172f69b62c8\") " pod="openshift-ingress/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq" Jul 17 07:47:27.901847 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:27.901605 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56a77e36-fa16-490b-b5bb-9172f69b62c8-istio-podinfo\") pod \"e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq\" (UID: \"56a77e36-fa16-490b-b5bb-9172f69b62c8\") " pod="openshift-ingress/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq" Jul 17 07:47:27.901847 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:27.901834 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56a77e36-fa16-490b-b5bb-9172f69b62c8-istio-data\") pod \"e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq\" (UID: \"56a77e36-fa16-490b-b5bb-9172f69b62c8\") " pod="openshift-ingress/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq" Jul 17 07:47:27.902020 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:27.901859 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56a77e36-fa16-490b-b5bb-9172f69b62c8-workload-socket\") pod \"e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq\" (UID: \"56a77e36-fa16-490b-b5bb-9172f69b62c8\") " pod="openshift-ingress/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq" Jul 17 07:47:27.902020 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:27.901957 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56a77e36-fa16-490b-b5bb-9172f69b62c8-credential-socket\") pod \"e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq\" (UID: \"56a77e36-fa16-490b-b5bb-9172f69b62c8\") " pod="openshift-ingress/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq" Jul 17 07:47:27.902129 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:27.902109 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56a77e36-fa16-490b-b5bb-9172f69b62c8-workload-certs\") pod \"e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq\" (UID: \"56a77e36-fa16-490b-b5bb-9172f69b62c8\") " pod="openshift-ingress/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq" Jul 17 07:47:27.902186 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:27.902141 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56a77e36-fa16-490b-b5bb-9172f69b62c8-istiod-ca-cert\") pod \"e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq\" (UID: \"56a77e36-fa16-490b-b5bb-9172f69b62c8\") " pod="openshift-ingress/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq" Jul 17 07:47:27.903805 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:27.903780 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56a77e36-fa16-490b-b5bb-9172f69b62c8-istio-envoy\") pod \"e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq\" (UID: \"56a77e36-fa16-490b-b5bb-9172f69b62c8\") " pod="openshift-ingress/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq" Jul 17 07:47:27.903915 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:27.903904 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56a77e36-fa16-490b-b5bb-9172f69b62c8-istio-podinfo\") pod \"e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq\" (UID: \"56a77e36-fa16-490b-b5bb-9172f69b62c8\") " pod="openshift-ingress/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq" Jul 17 07:47:27.912821 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:27.912757 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56a77e36-fa16-490b-b5bb-9172f69b62c8-istio-token\") pod \"e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq\" (UID: \"56a77e36-fa16-490b-b5bb-9172f69b62c8\") " pod="openshift-ingress/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq" Jul 17 07:47:27.912821 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:27.912781 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgj45\" (UniqueName: \"kubernetes.io/projected/56a77e36-fa16-490b-b5bb-9172f69b62c8-kube-api-access-xgj45\") pod \"e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq\" (UID: \"56a77e36-fa16-490b-b5bb-9172f69b62c8\") " pod="openshift-ingress/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq" Jul 17 07:47:27.982136 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:27.982109 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq" Jul 17 07:47:28.126573 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:28.126548 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq"] Jul 17 07:47:28.128795 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:47:28.128746 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56a77e36_fa16_490b_b5bb_9172f69b62c8.slice/crio-855ae88387c711591674d0833d31237902a74407783b98e04b702585389b2900 WatchSource:0}: Error finding container 855ae88387c711591674d0833d31237902a74407783b98e04b702585389b2900: Status 404 returned error can't find the container with id 855ae88387c711591674d0833d31237902a74407783b98e04b702585389b2900 Jul 17 07:47:28.130940 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:28.130887 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:47:28.131018 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:28.130981 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:47:28.131018 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:28.131014 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:47:28.692564 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:28.692523 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq" event={"ID":"56a77e36-fa16-490b-b5bb-9172f69b62c8","Type":"ContainerStarted","Data":"dbac265770f92265e0f7f74de49a978e374316d80c686429f22a283739310f8d"} Jul 17 07:47:28.692945 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:28.692593 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq" event={"ID":"56a77e36-fa16-490b-b5bb-9172f69b62c8","Type":"ContainerStarted","Data":"855ae88387c711591674d0833d31237902a74407783b98e04b702585389b2900"} Jul 17 07:47:28.716170 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:28.716120 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq" podStartSLOduration=1.716103959 podStartE2EDuration="1.716103959s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 07:47:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:47:28.716053091 +0000 UTC m=+2153.200742079" watchObservedRunningTime="2026-07-17 07:47:28.716103959 +0000 UTC m=+2153.200792932" Jul 17 07:47:28.983194 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:28.983118 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq" Jul 17 07:47:29.988091 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:29.988060 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq" Jul 17 07:47:30.704515 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:30.704484 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq" Jul 17 07:47:30.705494 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:30.705469 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq" Jul 17 07:47:39.755151 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:39.755069 2568 generic.go:356] "Generic (PLEG): container finished" podID="56a77e36-fa16-490b-b5bb-9172f69b62c8" containerID="dbac265770f92265e0f7f74de49a978e374316d80c686429f22a283739310f8d" exitCode=0 Jul 17 07:47:39.755575 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:39.755144 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq" event={"ID":"56a77e36-fa16-490b-b5bb-9172f69b62c8","Type":"ContainerDied","Data":"dbac265770f92265e0f7f74de49a978e374316d80c686429f22a283739310f8d"} Jul 17 07:47:39.755626 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:39.755592 2568 scope.go:117] "RemoveContainer" containerID="dbac265770f92265e0f7f74de49a978e374316d80c686429f22a283739310f8d" Jul 17 07:47:39.756574 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:39.756545 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:47:39.756637 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:39.756615 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:47:39.756684 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:39.756650 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:47:40.762397 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:40.762362 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq" event={"ID":"56a77e36-fa16-490b-b5bb-9172f69b62c8","Type":"ContainerStarted","Data":"8e53bf1a0a989bbe6f411c6f97da63ef4c749a766f1c529113c61f28cfcd9b44"} Jul 17 07:47:40.983705 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:40.983671 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq" Jul 17 07:47:40.984127 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:40.984103 2568 patch_prober.go:28] interesting pod/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 17 07:47:40.984321 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:40.984276 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq" podUID="56a77e36-fa16-490b-b5bb-9172f69b62c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 17 07:47:41.982873 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:41.982835 2568 patch_prober.go:28] interesting pod/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 17 07:47:41.983525 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:41.983479 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq" podUID="56a77e36-fa16-490b-b5bb-9172f69b62c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 17 07:47:42.983274 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:42.983237 2568 patch_prober.go:28] interesting pod/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 17 07:47:42.983844 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:42.983814 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq" podUID="56a77e36-fa16-490b-b5bb-9172f69b62c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 17 07:47:43.982742 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:43.982702 2568 patch_prober.go:28] interesting pod/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 17 07:47:43.983049 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:43.983018 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq" podUID="56a77e36-fa16-490b-b5bb-9172f69b62c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 17 07:47:44.987540 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:44.987509 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq" Jul 17 07:47:44.987958 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:44.987867 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq" Jul 17 07:47:44.988653 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:47:44.988633 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq" Jul 17 07:50:33.371041 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:50:33.370949 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq"] Jul 17 07:50:33.371618 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:50:33.371196 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq" podUID="56a77e36-fa16-490b-b5bb-9172f69b62c8" containerName="istio-proxy" containerID="cri-o://8e53bf1a0a989bbe6f411c6f97da63ef4c749a766f1c529113c61f28cfcd9b44" gracePeriod=30 Jul 17 07:50:38.631454 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:50:38.631423 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq" Jul 17 07:50:38.725089 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:50:38.725052 2568 generic.go:356] "Generic (PLEG): container finished" podID="56a77e36-fa16-490b-b5bb-9172f69b62c8" containerID="8e53bf1a0a989bbe6f411c6f97da63ef4c749a766f1c529113c61f28cfcd9b44" exitCode=0 Jul 17 07:50:38.725250 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:50:38.725104 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq" Jul 17 07:50:38.725250 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:50:38.725096 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq" event={"ID":"56a77e36-fa16-490b-b5bb-9172f69b62c8","Type":"ContainerDied","Data":"8e53bf1a0a989bbe6f411c6f97da63ef4c749a766f1c529113c61f28cfcd9b44"} Jul 17 07:50:38.725250 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:50:38.725178 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq" event={"ID":"56a77e36-fa16-490b-b5bb-9172f69b62c8","Type":"ContainerDied","Data":"855ae88387c711591674d0833d31237902a74407783b98e04b702585389b2900"} Jul 17 07:50:38.725250 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:50:38.725209 2568 scope.go:117] "RemoveContainer" containerID="8e53bf1a0a989bbe6f411c6f97da63ef4c749a766f1c529113c61f28cfcd9b44" Jul 17 07:50:38.731645 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:50:38.731618 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56a77e36-fa16-490b-b5bb-9172f69b62c8-istio-data\") pod \"56a77e36-fa16-490b-b5bb-9172f69b62c8\" (UID: \"56a77e36-fa16-490b-b5bb-9172f69b62c8\") " Jul 17 07:50:38.731772 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:50:38.731692 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56a77e36-fa16-490b-b5bb-9172f69b62c8-workload-socket\") pod \"56a77e36-fa16-490b-b5bb-9172f69b62c8\" (UID: \"56a77e36-fa16-490b-b5bb-9172f69b62c8\") " Jul 17 07:50:38.731772 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:50:38.731723 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56a77e36-fa16-490b-b5bb-9172f69b62c8-istio-token\") pod \"56a77e36-fa16-490b-b5bb-9172f69b62c8\" (UID: \"56a77e36-fa16-490b-b5bb-9172f69b62c8\") " Jul 17 07:50:38.731772 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:50:38.731746 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56a77e36-fa16-490b-b5bb-9172f69b62c8-credential-socket\") pod \"56a77e36-fa16-490b-b5bb-9172f69b62c8\" (UID: \"56a77e36-fa16-490b-b5bb-9172f69b62c8\") " Jul 17 07:50:38.731955 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:50:38.731822 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56a77e36-fa16-490b-b5bb-9172f69b62c8-istiod-ca-cert\") pod \"56a77e36-fa16-490b-b5bb-9172f69b62c8\" (UID: \"56a77e36-fa16-490b-b5bb-9172f69b62c8\") " Jul 17 07:50:38.731955 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:50:38.731850 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56a77e36-fa16-490b-b5bb-9172f69b62c8-istio-envoy\") pod \"56a77e36-fa16-490b-b5bb-9172f69b62c8\" (UID: \"56a77e36-fa16-490b-b5bb-9172f69b62c8\") " Jul 17 07:50:38.731955 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:50:38.731880 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xgj45\" (UniqueName: \"kubernetes.io/projected/56a77e36-fa16-490b-b5bb-9172f69b62c8-kube-api-access-xgj45\") pod \"56a77e36-fa16-490b-b5bb-9172f69b62c8\" (UID: \"56a77e36-fa16-490b-b5bb-9172f69b62c8\") " Jul 17 07:50:38.731955 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:50:38.731911 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56a77e36-fa16-490b-b5bb-9172f69b62c8-workload-certs\") pod \"56a77e36-fa16-490b-b5bb-9172f69b62c8\" (UID: \"56a77e36-fa16-490b-b5bb-9172f69b62c8\") " Jul 17 07:50:38.731955 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:50:38.731929 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56a77e36-fa16-490b-b5bb-9172f69b62c8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "56a77e36-fa16-490b-b5bb-9172f69b62c8" (UID: "56a77e36-fa16-490b-b5bb-9172f69b62c8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:50:38.732229 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:50:38.731964 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56a77e36-fa16-490b-b5bb-9172f69b62c8-istio-podinfo\") pod \"56a77e36-fa16-490b-b5bb-9172f69b62c8\" (UID: \"56a77e36-fa16-490b-b5bb-9172f69b62c8\") " Jul 17 07:50:38.732332 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:50:38.732278 2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56a77e36-fa16-490b-b5bb-9172f69b62c8-workload-socket\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:50:38.732332 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:50:38.732305 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56a77e36-fa16-490b-b5bb-9172f69b62c8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "56a77e36-fa16-490b-b5bb-9172f69b62c8" (UID: "56a77e36-fa16-490b-b5bb-9172f69b62c8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 07:50:38.733668 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:50:38.732586 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56a77e36-fa16-490b-b5bb-9172f69b62c8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "56a77e36-fa16-490b-b5bb-9172f69b62c8" (UID: "56a77e36-fa16-490b-b5bb-9172f69b62c8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:50:38.733668 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:50:38.732637 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56a77e36-fa16-490b-b5bb-9172f69b62c8-istio-data" (OuterVolumeSpecName: "istio-data") pod "56a77e36-fa16-490b-b5bb-9172f69b62c8" (UID: "56a77e36-fa16-490b-b5bb-9172f69b62c8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:50:38.733668 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:50:38.732651 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56a77e36-fa16-490b-b5bb-9172f69b62c8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "56a77e36-fa16-490b-b5bb-9172f69b62c8" (UID: "56a77e36-fa16-490b-b5bb-9172f69b62c8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:50:38.734244 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:50:38.734217 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56a77e36-fa16-490b-b5bb-9172f69b62c8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "56a77e36-fa16-490b-b5bb-9172f69b62c8" (UID: "56a77e36-fa16-490b-b5bb-9172f69b62c8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:50:38.734378 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:50:38.734346 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/56a77e36-fa16-490b-b5bb-9172f69b62c8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "56a77e36-fa16-490b-b5bb-9172f69b62c8" (UID: "56a77e36-fa16-490b-b5bb-9172f69b62c8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 17 07:50:38.734666 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:50:38.734641 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56a77e36-fa16-490b-b5bb-9172f69b62c8-istio-token" (OuterVolumeSpecName: "istio-token") pod "56a77e36-fa16-490b-b5bb-9172f69b62c8" (UID: "56a77e36-fa16-490b-b5bb-9172f69b62c8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 07:50:38.734666 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:50:38.734651 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56a77e36-fa16-490b-b5bb-9172f69b62c8-kube-api-access-xgj45" (OuterVolumeSpecName: "kube-api-access-xgj45") pod "56a77e36-fa16-490b-b5bb-9172f69b62c8" (UID: "56a77e36-fa16-490b-b5bb-9172f69b62c8"). InnerVolumeSpecName "kube-api-access-xgj45". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 07:50:38.736146 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:50:38.736124 2568 scope.go:117] "RemoveContainer" containerID="dbac265770f92265e0f7f74de49a978e374316d80c686429f22a283739310f8d" Jul 17 07:50:38.753344 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:50:38.753324 2568 scope.go:117] "RemoveContainer" containerID="8e53bf1a0a989bbe6f411c6f97da63ef4c749a766f1c529113c61f28cfcd9b44" Jul 17 07:50:38.753618 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:50:38.753593 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e53bf1a0a989bbe6f411c6f97da63ef4c749a766f1c529113c61f28cfcd9b44\": container with ID starting with 8e53bf1a0a989bbe6f411c6f97da63ef4c749a766f1c529113c61f28cfcd9b44 not found: ID does not exist" containerID="8e53bf1a0a989bbe6f411c6f97da63ef4c749a766f1c529113c61f28cfcd9b44" Jul 17 07:50:38.753662 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:50:38.753628 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e53bf1a0a989bbe6f411c6f97da63ef4c749a766f1c529113c61f28cfcd9b44"} err="failed to get container status \"8e53bf1a0a989bbe6f411c6f97da63ef4c749a766f1c529113c61f28cfcd9b44\": rpc error: code = NotFound desc = could not find container \"8e53bf1a0a989bbe6f411c6f97da63ef4c749a766f1c529113c61f28cfcd9b44\": container with ID starting with 8e53bf1a0a989bbe6f411c6f97da63ef4c749a766f1c529113c61f28cfcd9b44 not found: ID does not exist" Jul 17 07:50:38.753662 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:50:38.753647 2568 scope.go:117] "RemoveContainer" containerID="dbac265770f92265e0f7f74de49a978e374316d80c686429f22a283739310f8d" Jul 17 07:50:38.753918 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:50:38.753901 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dbac265770f92265e0f7f74de49a978e374316d80c686429f22a283739310f8d\": container with ID starting with dbac265770f92265e0f7f74de49a978e374316d80c686429f22a283739310f8d not found: ID does not exist" containerID="dbac265770f92265e0f7f74de49a978e374316d80c686429f22a283739310f8d" Jul 17 07:50:38.753958 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:50:38.753927 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbac265770f92265e0f7f74de49a978e374316d80c686429f22a283739310f8d"} err="failed to get container status \"dbac265770f92265e0f7f74de49a978e374316d80c686429f22a283739310f8d\": rpc error: code = NotFound desc = could not find container \"dbac265770f92265e0f7f74de49a978e374316d80c686429f22a283739310f8d\": container with ID starting with dbac265770f92265e0f7f74de49a978e374316d80c686429f22a283739310f8d not found: ID does not exist" Jul 17 07:50:38.833249 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:50:38.833226 2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56a77e36-fa16-490b-b5bb-9172f69b62c8-istio-token\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:50:38.833249 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:50:38.833249 2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56a77e36-fa16-490b-b5bb-9172f69b62c8-credential-socket\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:50:38.833416 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:50:38.833259 2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56a77e36-fa16-490b-b5bb-9172f69b62c8-istiod-ca-cert\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:50:38.833416 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:50:38.833268 2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56a77e36-fa16-490b-b5bb-9172f69b62c8-istio-envoy\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:50:38.833416 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:50:38.833276 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xgj45\" (UniqueName: \"kubernetes.io/projected/56a77e36-fa16-490b-b5bb-9172f69b62c8-kube-api-access-xgj45\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:50:38.833416 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:50:38.833298 2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56a77e36-fa16-490b-b5bb-9172f69b62c8-workload-certs\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:50:38.833416 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:50:38.833307 2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56a77e36-fa16-490b-b5bb-9172f69b62c8-istio-podinfo\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:50:38.833416 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:50:38.833314 2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56a77e36-fa16-490b-b5bb-9172f69b62c8-istio-data\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:50:39.049235 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:50:39.049204 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq"] Jul 17 07:50:39.054878 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:50:39.054853 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-f44a26e0-openshift-default-9ffdc6f9-7d9tq"] Jul 17 07:50:40.000538 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:50:40.000503 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56a77e36-fa16-490b-b5bb-9172f69b62c8" path="/var/lib/kubelet/pods/56a77e36-fa16-490b-b5bb-9172f69b62c8/volumes" Jul 17 07:51:36.210233 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:51:36.210197 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-xr6pg_2ce00a54-15e9-42b7-b3a8-9e58d1101558/cluster-monitoring-operator/0.log" Jul 17 07:51:36.215809 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:51:36.215790 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-xpxv5_27538d1e-992b-4651-a2d2-7a7ecf442806/console-operator/1.log" Jul 17 07:51:36.222866 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:51:36.222843 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cpwz4_1c44a3ae-4c83-4203-8b0b-116d6618accf/ovn-acl-logging/0.log" Jul 17 07:51:36.223915 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:51:36.223895 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-xr6pg_2ce00a54-15e9-42b7-b3a8-9e58d1101558/cluster-monitoring-operator/0.log" Jul 17 07:51:36.228714 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:51:36.228695 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-xpxv5_27538d1e-992b-4651-a2d2-7a7ecf442806/console-operator/1.log" Jul 17 07:51:36.231894 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:51:36.231878 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cpwz4_1c44a3ae-4c83-4203-8b0b-116d6618accf/ovn-acl-logging/0.log" Jul 17 07:52:24.728017 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:24.727986 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl"] Jul 17 07:52:24.728571 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:24.728533 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56a77e36-fa16-490b-b5bb-9172f69b62c8" containerName="istio-proxy" Jul 17 07:52:24.728571 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:24.728555 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="56a77e36-fa16-490b-b5bb-9172f69b62c8" containerName="istio-proxy" Jul 17 07:52:24.728669 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:24.728573 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56a77e36-fa16-490b-b5bb-9172f69b62c8" containerName="istio-proxy" Jul 17 07:52:24.728669 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:24.728582 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="56a77e36-fa16-490b-b5bb-9172f69b62c8" containerName="istio-proxy" Jul 17 07:52:24.728743 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:24.728675 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="85a5bcef-6272-40c2-9bb9-836ee75525f2" containerName="revoke-keys" Jul 17 07:52:24.728743 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:24.728688 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="56a77e36-fa16-490b-b5bb-9172f69b62c8" containerName="istio-proxy" Jul 17 07:52:24.728743 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:24.728699 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="56a77e36-fa16-490b-b5bb-9172f69b62c8" containerName="istio-proxy" Jul 17 07:52:24.731763 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:24.731747 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl" Jul 17 07:52:24.734635 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:24.734614 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-f88ba1e3-openshift-default-dockercfg-hg6mp\"" Jul 17 07:52:24.745088 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:24.745066 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl"] Jul 17 07:52:24.755847 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:24.755821 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3825789-59b7-4e9c-86ff-b207b8335ac4-workload-socket\") pod \"e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl\" (UID: \"e3825789-59b7-4e9c-86ff-b207b8335ac4\") " pod="openshift-ingress/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl" Jul 17 07:52:24.755970 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:24.755882 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3825789-59b7-4e9c-86ff-b207b8335ac4-istiod-ca-cert\") pod \"e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl\" (UID: \"e3825789-59b7-4e9c-86ff-b207b8335ac4\") " pod="openshift-ingress/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl" Jul 17 07:52:24.755970 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:24.755916 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3825789-59b7-4e9c-86ff-b207b8335ac4-istio-envoy\") pod \"e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl\" (UID: \"e3825789-59b7-4e9c-86ff-b207b8335ac4\") " pod="openshift-ingress/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl" Jul 17 07:52:24.756048 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:24.756012 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3825789-59b7-4e9c-86ff-b207b8335ac4-credential-socket\") pod \"e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl\" (UID: \"e3825789-59b7-4e9c-86ff-b207b8335ac4\") " pod="openshift-ingress/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl" Jul 17 07:52:24.756048 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:24.756039 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ft24c\" (UniqueName: \"kubernetes.io/projected/e3825789-59b7-4e9c-86ff-b207b8335ac4-kube-api-access-ft24c\") pod \"e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl\" (UID: \"e3825789-59b7-4e9c-86ff-b207b8335ac4\") " pod="openshift-ingress/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl" Jul 17 07:52:24.756127 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:24.756074 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3825789-59b7-4e9c-86ff-b207b8335ac4-istio-podinfo\") pod \"e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl\" (UID: \"e3825789-59b7-4e9c-86ff-b207b8335ac4\") " pod="openshift-ingress/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl" Jul 17 07:52:24.756165 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:24.756142 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3825789-59b7-4e9c-86ff-b207b8335ac4-workload-certs\") pod \"e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl\" (UID: \"e3825789-59b7-4e9c-86ff-b207b8335ac4\") " pod="openshift-ingress/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl" Jul 17 07:52:24.756208 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:24.756167 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3825789-59b7-4e9c-86ff-b207b8335ac4-istio-token\") pod \"e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl\" (UID: \"e3825789-59b7-4e9c-86ff-b207b8335ac4\") " pod="openshift-ingress/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl" Jul 17 07:52:24.756208 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:24.756189 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3825789-59b7-4e9c-86ff-b207b8335ac4-istio-data\") pod \"e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl\" (UID: \"e3825789-59b7-4e9c-86ff-b207b8335ac4\") " pod="openshift-ingress/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl" Jul 17 07:52:24.856887 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:24.856852 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3825789-59b7-4e9c-86ff-b207b8335ac4-istiod-ca-cert\") pod \"e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl\" (UID: \"e3825789-59b7-4e9c-86ff-b207b8335ac4\") " pod="openshift-ingress/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl" Jul 17 07:52:24.857091 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:24.856903 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3825789-59b7-4e9c-86ff-b207b8335ac4-istio-envoy\") pod \"e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl\" (UID: \"e3825789-59b7-4e9c-86ff-b207b8335ac4\") " pod="openshift-ingress/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl" Jul 17 07:52:24.857091 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:24.856946 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3825789-59b7-4e9c-86ff-b207b8335ac4-credential-socket\") pod \"e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl\" (UID: \"e3825789-59b7-4e9c-86ff-b207b8335ac4\") " pod="openshift-ingress/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl" Jul 17 07:52:24.857091 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:24.856963 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ft24c\" (UniqueName: \"kubernetes.io/projected/e3825789-59b7-4e9c-86ff-b207b8335ac4-kube-api-access-ft24c\") pod \"e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl\" (UID: \"e3825789-59b7-4e9c-86ff-b207b8335ac4\") " pod="openshift-ingress/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl" Jul 17 07:52:24.857091 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:24.856998 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3825789-59b7-4e9c-86ff-b207b8335ac4-istio-podinfo\") pod \"e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl\" (UID: \"e3825789-59b7-4e9c-86ff-b207b8335ac4\") " pod="openshift-ingress/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl" Jul 17 07:52:24.857091 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:24.857056 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3825789-59b7-4e9c-86ff-b207b8335ac4-workload-certs\") pod \"e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl\" (UID: \"e3825789-59b7-4e9c-86ff-b207b8335ac4\") " pod="openshift-ingress/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl" Jul 17 07:52:24.857091 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:24.857083 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3825789-59b7-4e9c-86ff-b207b8335ac4-istio-token\") pod \"e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl\" (UID: \"e3825789-59b7-4e9c-86ff-b207b8335ac4\") " pod="openshift-ingress/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl" Jul 17 07:52:24.857494 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:24.857109 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3825789-59b7-4e9c-86ff-b207b8335ac4-istio-data\") pod \"e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl\" (UID: \"e3825789-59b7-4e9c-86ff-b207b8335ac4\") " pod="openshift-ingress/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl" Jul 17 07:52:24.857494 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:24.857159 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3825789-59b7-4e9c-86ff-b207b8335ac4-workload-socket\") pod \"e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl\" (UID: \"e3825789-59b7-4e9c-86ff-b207b8335ac4\") " pod="openshift-ingress/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl" Jul 17 07:52:24.857637 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:24.857612 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3825789-59b7-4e9c-86ff-b207b8335ac4-istiod-ca-cert\") pod \"e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl\" (UID: \"e3825789-59b7-4e9c-86ff-b207b8335ac4\") " pod="openshift-ingress/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl" Jul 17 07:52:24.857766 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:24.857743 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3825789-59b7-4e9c-86ff-b207b8335ac4-workload-socket\") pod \"e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl\" (UID: \"e3825789-59b7-4e9c-86ff-b207b8335ac4\") " pod="openshift-ingress/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl" Jul 17 07:52:24.857868 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:24.857745 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3825789-59b7-4e9c-86ff-b207b8335ac4-credential-socket\") pod \"e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl\" (UID: \"e3825789-59b7-4e9c-86ff-b207b8335ac4\") " pod="openshift-ingress/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl" Jul 17 07:52:24.857932 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:24.857862 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3825789-59b7-4e9c-86ff-b207b8335ac4-istio-data\") pod \"e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl\" (UID: \"e3825789-59b7-4e9c-86ff-b207b8335ac4\") " pod="openshift-ingress/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl" Jul 17 07:52:24.860556 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:24.859820 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3825789-59b7-4e9c-86ff-b207b8335ac4-workload-certs\") pod \"e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl\" (UID: \"e3825789-59b7-4e9c-86ff-b207b8335ac4\") " pod="openshift-ingress/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl" Jul 17 07:52:24.861555 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:24.861443 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3825789-59b7-4e9c-86ff-b207b8335ac4-istio-podinfo\") pod \"e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl\" (UID: \"e3825789-59b7-4e9c-86ff-b207b8335ac4\") " pod="openshift-ingress/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl" Jul 17 07:52:24.866660 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:24.866631 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3825789-59b7-4e9c-86ff-b207b8335ac4-istio-envoy\") pod \"e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl\" (UID: \"e3825789-59b7-4e9c-86ff-b207b8335ac4\") " pod="openshift-ingress/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl" Jul 17 07:52:24.868760 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:24.868730 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3825789-59b7-4e9c-86ff-b207b8335ac4-istio-token\") pod \"e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl\" (UID: \"e3825789-59b7-4e9c-86ff-b207b8335ac4\") " pod="openshift-ingress/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl" Jul 17 07:52:24.873739 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:24.873713 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ft24c\" (UniqueName: \"kubernetes.io/projected/e3825789-59b7-4e9c-86ff-b207b8335ac4-kube-api-access-ft24c\") pod \"e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl\" (UID: \"e3825789-59b7-4e9c-86ff-b207b8335ac4\") " pod="openshift-ingress/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl" Jul 17 07:52:25.041414 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:25.041322 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl" Jul 17 07:52:25.186712 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:25.186688 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl"] Jul 17 07:52:25.188046 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:52:25.188013 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3825789_59b7_4e9c_86ff_b207b8335ac4.slice/crio-fdc2c7bc5bfe69c60a4745a6687494c94306daebbdbe7b3fdfd55e93cdb7256c WatchSource:0}: Error finding container fdc2c7bc5bfe69c60a4745a6687494c94306daebbdbe7b3fdfd55e93cdb7256c: Status 404 returned error can't find the container with id fdc2c7bc5bfe69c60a4745a6687494c94306daebbdbe7b3fdfd55e93cdb7256c Jul 17 07:52:25.190198 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:25.190161 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:52:25.190301 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:25.190227 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:52:25.190301 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:25.190256 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:52:25.311146 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:25.311054 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl" event={"ID":"e3825789-59b7-4e9c-86ff-b207b8335ac4","Type":"ContainerStarted","Data":"77d1418ed3582fc4acd0ce7a18f5c6417f7220f1129430336b7eabc9b6ee26f7"} Jul 17 07:52:25.311146 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:25.311115 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl" event={"ID":"e3825789-59b7-4e9c-86ff-b207b8335ac4","Type":"ContainerStarted","Data":"fdc2c7bc5bfe69c60a4745a6687494c94306daebbdbe7b3fdfd55e93cdb7256c"} Jul 17 07:52:25.358889 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:25.358828 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl" podStartSLOduration=1.358805343 podStartE2EDuration="1.358805343s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 07:52:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:52:25.353893676 +0000 UTC m=+2449.838582651" watchObservedRunningTime="2026-07-17 07:52:25.358805343 +0000 UTC m=+2449.843494318" Jul 17 07:52:26.041598 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:26.041570 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl" Jul 17 07:52:26.046430 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:26.046408 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl" Jul 17 07:52:26.316694 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:26.316621 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl" Jul 17 07:52:26.317645 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:26.317625 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl" Jul 17 07:52:41.342728 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:41.341530 2568 patch_prober.go:28] interesting pod/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": read tcp 10.132.0.2:47594->10.132.0.62:15021: read: connection reset by peer" start-of-body= Jul 17 07:52:41.342728 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:41.341605 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl" podUID="e3825789-59b7-4e9c-86ff-b207b8335ac4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": read tcp 10.132.0.2:47594->10.132.0.62:15021: read: connection reset by peer" Jul 17 07:52:41.411218 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:41.411190 2568 generic.go:356] "Generic (PLEG): container finished" podID="e3825789-59b7-4e9c-86ff-b207b8335ac4" containerID="77d1418ed3582fc4acd0ce7a18f5c6417f7220f1129430336b7eabc9b6ee26f7" exitCode=0 Jul 17 07:52:41.411359 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:41.411258 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl" event={"ID":"e3825789-59b7-4e9c-86ff-b207b8335ac4","Type":"ContainerDied","Data":"77d1418ed3582fc4acd0ce7a18f5c6417f7220f1129430336b7eabc9b6ee26f7"} Jul 17 07:52:41.411646 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:41.411633 2568 scope.go:117] "RemoveContainer" containerID="77d1418ed3582fc4acd0ce7a18f5c6417f7220f1129430336b7eabc9b6ee26f7" Jul 17 07:52:41.412770 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:41.412734 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:52:41.412870 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:41.412833 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:52:41.412909 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:41.412883 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:52:42.418323 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:42.418264 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl" event={"ID":"e3825789-59b7-4e9c-86ff-b207b8335ac4","Type":"ContainerStarted","Data":"463b68765c380745ec38181810d96ec15e9e703e954108d92bb1d364b12edff6"} Jul 17 07:52:43.041602 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:43.041565 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl" Jul 17 07:52:43.041880 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:43.041857 2568 patch_prober.go:28] interesting pod/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 17 07:52:43.041930 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:43.041904 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl" podUID="e3825789-59b7-4e9c-86ff-b207b8335ac4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 17 07:52:44.042468 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:44.042434 2568 patch_prober.go:28] interesting pod/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 17 07:52:44.042849 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:44.042484 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl" podUID="e3825789-59b7-4e9c-86ff-b207b8335ac4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 17 07:52:45.042486 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:45.042444 2568 patch_prober.go:28] interesting pod/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 17 07:52:45.042977 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:45.042512 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl" podUID="e3825789-59b7-4e9c-86ff-b207b8335ac4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 17 07:52:46.041835 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:46.041795 2568 patch_prober.go:28] interesting pod/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 17 07:52:46.042023 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:46.041858 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl" podUID="e3825789-59b7-4e9c-86ff-b207b8335ac4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 17 07:52:47.045910 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:47.045879 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl" Jul 17 07:52:47.046334 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:47.046148 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl" Jul 17 07:52:47.046873 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:52:47.046856 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl" Jul 17 07:53:06.217862 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:06.217825 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl"] Jul 17 07:53:06.218471 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:06.218033 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl" podUID="e3825789-59b7-4e9c-86ff-b207b8335ac4" containerName="istio-proxy" containerID="cri-o://463b68765c380745ec38181810d96ec15e9e703e954108d92bb1d364b12edff6" gracePeriod=30 Jul 17 07:53:11.471214 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:11.471190 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl" Jul 17 07:53:11.483452 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:11.483400 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3825789-59b7-4e9c-86ff-b207b8335ac4-workload-socket\") pod \"e3825789-59b7-4e9c-86ff-b207b8335ac4\" (UID: \"e3825789-59b7-4e9c-86ff-b207b8335ac4\") " Jul 17 07:53:11.483452 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:11.483429 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3825789-59b7-4e9c-86ff-b207b8335ac4-istio-envoy\") pod \"e3825789-59b7-4e9c-86ff-b207b8335ac4\" (UID: \"e3825789-59b7-4e9c-86ff-b207b8335ac4\") " Jul 17 07:53:11.483452 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:11.483448 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3825789-59b7-4e9c-86ff-b207b8335ac4-istio-podinfo\") pod \"e3825789-59b7-4e9c-86ff-b207b8335ac4\" (UID: \"e3825789-59b7-4e9c-86ff-b207b8335ac4\") " Jul 17 07:53:11.483673 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:11.483482 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3825789-59b7-4e9c-86ff-b207b8335ac4-workload-certs\") pod \"e3825789-59b7-4e9c-86ff-b207b8335ac4\" (UID: \"e3825789-59b7-4e9c-86ff-b207b8335ac4\") " Jul 17 07:53:11.483673 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:11.483547 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3825789-59b7-4e9c-86ff-b207b8335ac4-credential-socket\") pod \"e3825789-59b7-4e9c-86ff-b207b8335ac4\" (UID: \"e3825789-59b7-4e9c-86ff-b207b8335ac4\") " Jul 17 07:53:11.483673 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:11.483576 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ft24c\" (UniqueName: \"kubernetes.io/projected/e3825789-59b7-4e9c-86ff-b207b8335ac4-kube-api-access-ft24c\") pod \"e3825789-59b7-4e9c-86ff-b207b8335ac4\" (UID: \"e3825789-59b7-4e9c-86ff-b207b8335ac4\") " Jul 17 07:53:11.483673 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:11.483614 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3825789-59b7-4e9c-86ff-b207b8335ac4-istio-token\") pod \"e3825789-59b7-4e9c-86ff-b207b8335ac4\" (UID: \"e3825789-59b7-4e9c-86ff-b207b8335ac4\") " Jul 17 07:53:11.483673 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:11.483653 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3825789-59b7-4e9c-86ff-b207b8335ac4-istiod-ca-cert\") pod \"e3825789-59b7-4e9c-86ff-b207b8335ac4\" (UID: \"e3825789-59b7-4e9c-86ff-b207b8335ac4\") " Jul 17 07:53:11.483939 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:11.483688 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3825789-59b7-4e9c-86ff-b207b8335ac4-istio-data\") pod \"e3825789-59b7-4e9c-86ff-b207b8335ac4\" (UID: \"e3825789-59b7-4e9c-86ff-b207b8335ac4\") " Jul 17 07:53:11.483939 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:11.483688 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3825789-59b7-4e9c-86ff-b207b8335ac4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e3825789-59b7-4e9c-86ff-b207b8335ac4" (UID: "e3825789-59b7-4e9c-86ff-b207b8335ac4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:53:11.484047 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:11.483839 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3825789-59b7-4e9c-86ff-b207b8335ac4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e3825789-59b7-4e9c-86ff-b207b8335ac4" (UID: "e3825789-59b7-4e9c-86ff-b207b8335ac4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:53:11.484047 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:11.483847 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3825789-59b7-4e9c-86ff-b207b8335ac4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e3825789-59b7-4e9c-86ff-b207b8335ac4" (UID: "e3825789-59b7-4e9c-86ff-b207b8335ac4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:53:11.484047 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:11.484035 2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3825789-59b7-4e9c-86ff-b207b8335ac4-credential-socket\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:53:11.484208 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:11.484057 2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3825789-59b7-4e9c-86ff-b207b8335ac4-workload-socket\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:53:11.484208 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:11.484074 2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3825789-59b7-4e9c-86ff-b207b8335ac4-workload-certs\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:53:11.484486 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:11.484456 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3825789-59b7-4e9c-86ff-b207b8335ac4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e3825789-59b7-4e9c-86ff-b207b8335ac4" (UID: "e3825789-59b7-4e9c-86ff-b207b8335ac4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 07:53:11.484634 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:11.484572 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3825789-59b7-4e9c-86ff-b207b8335ac4-istio-data" (OuterVolumeSpecName: "istio-data") pod "e3825789-59b7-4e9c-86ff-b207b8335ac4" (UID: "e3825789-59b7-4e9c-86ff-b207b8335ac4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:53:11.485798 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:11.485752 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3825789-59b7-4e9c-86ff-b207b8335ac4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e3825789-59b7-4e9c-86ff-b207b8335ac4" (UID: "e3825789-59b7-4e9c-86ff-b207b8335ac4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:53:11.485798 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:11.485771 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e3825789-59b7-4e9c-86ff-b207b8335ac4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e3825789-59b7-4e9c-86ff-b207b8335ac4" (UID: "e3825789-59b7-4e9c-86ff-b207b8335ac4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 17 07:53:11.485960 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:11.485752 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3825789-59b7-4e9c-86ff-b207b8335ac4-istio-token" (OuterVolumeSpecName: "istio-token") pod "e3825789-59b7-4e9c-86ff-b207b8335ac4" (UID: "e3825789-59b7-4e9c-86ff-b207b8335ac4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 07:53:11.486053 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:11.486022 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3825789-59b7-4e9c-86ff-b207b8335ac4-kube-api-access-ft24c" (OuterVolumeSpecName: "kube-api-access-ft24c") pod "e3825789-59b7-4e9c-86ff-b207b8335ac4" (UID: "e3825789-59b7-4e9c-86ff-b207b8335ac4"). InnerVolumeSpecName "kube-api-access-ft24c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 07:53:11.577184 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:11.577143 2568 generic.go:356] "Generic (PLEG): container finished" podID="e3825789-59b7-4e9c-86ff-b207b8335ac4" containerID="463b68765c380745ec38181810d96ec15e9e703e954108d92bb1d364b12edff6" exitCode=0 Jul 17 07:53:11.577350 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:11.577190 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl" event={"ID":"e3825789-59b7-4e9c-86ff-b207b8335ac4","Type":"ContainerDied","Data":"463b68765c380745ec38181810d96ec15e9e703e954108d92bb1d364b12edff6"} Jul 17 07:53:11.577350 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:11.577257 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl" event={"ID":"e3825789-59b7-4e9c-86ff-b207b8335ac4","Type":"ContainerDied","Data":"fdc2c7bc5bfe69c60a4745a6687494c94306daebbdbe7b3fdfd55e93cdb7256c"} Jul 17 07:53:11.577350 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:11.577192 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl" Jul 17 07:53:11.577350 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:11.577312 2568 scope.go:117] "RemoveContainer" containerID="463b68765c380745ec38181810d96ec15e9e703e954108d92bb1d364b12edff6" Jul 17 07:53:11.585263 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:11.585006 2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3825789-59b7-4e9c-86ff-b207b8335ac4-istio-envoy\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:53:11.585263 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:11.585024 2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3825789-59b7-4e9c-86ff-b207b8335ac4-istio-podinfo\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:53:11.585263 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:11.585033 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ft24c\" (UniqueName: \"kubernetes.io/projected/e3825789-59b7-4e9c-86ff-b207b8335ac4-kube-api-access-ft24c\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:53:11.585263 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:11.585042 2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3825789-59b7-4e9c-86ff-b207b8335ac4-istio-token\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:53:11.585263 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:11.585051 2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3825789-59b7-4e9c-86ff-b207b8335ac4-istiod-ca-cert\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:53:11.585263 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:11.585063 2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3825789-59b7-4e9c-86ff-b207b8335ac4-istio-data\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:53:11.587119 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:11.587099 2568 scope.go:117] "RemoveContainer" containerID="77d1418ed3582fc4acd0ce7a18f5c6417f7220f1129430336b7eabc9b6ee26f7" Jul 17 07:53:11.595627 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:11.595611 2568 scope.go:117] "RemoveContainer" containerID="463b68765c380745ec38181810d96ec15e9e703e954108d92bb1d364b12edff6" Jul 17 07:53:11.595902 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:53:11.595883 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"463b68765c380745ec38181810d96ec15e9e703e954108d92bb1d364b12edff6\": container with ID starting with 463b68765c380745ec38181810d96ec15e9e703e954108d92bb1d364b12edff6 not found: ID does not exist" containerID="463b68765c380745ec38181810d96ec15e9e703e954108d92bb1d364b12edff6" Jul 17 07:53:11.595953 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:11.595910 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"463b68765c380745ec38181810d96ec15e9e703e954108d92bb1d364b12edff6"} err="failed to get container status \"463b68765c380745ec38181810d96ec15e9e703e954108d92bb1d364b12edff6\": rpc error: code = NotFound desc = could not find container \"463b68765c380745ec38181810d96ec15e9e703e954108d92bb1d364b12edff6\": container with ID starting with 463b68765c380745ec38181810d96ec15e9e703e954108d92bb1d364b12edff6 not found: ID does not exist" Jul 17 07:53:11.595953 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:11.595927 2568 scope.go:117] "RemoveContainer" containerID="77d1418ed3582fc4acd0ce7a18f5c6417f7220f1129430336b7eabc9b6ee26f7" Jul 17 07:53:11.596144 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:53:11.596123 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77d1418ed3582fc4acd0ce7a18f5c6417f7220f1129430336b7eabc9b6ee26f7\": container with ID starting with 77d1418ed3582fc4acd0ce7a18f5c6417f7220f1129430336b7eabc9b6ee26f7 not found: ID does not exist" containerID="77d1418ed3582fc4acd0ce7a18f5c6417f7220f1129430336b7eabc9b6ee26f7" Jul 17 07:53:11.596202 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:11.596154 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77d1418ed3582fc4acd0ce7a18f5c6417f7220f1129430336b7eabc9b6ee26f7"} err="failed to get container status \"77d1418ed3582fc4acd0ce7a18f5c6417f7220f1129430336b7eabc9b6ee26f7\": rpc error: code = NotFound desc = could not find container \"77d1418ed3582fc4acd0ce7a18f5c6417f7220f1129430336b7eabc9b6ee26f7\": container with ID starting with 77d1418ed3582fc4acd0ce7a18f5c6417f7220f1129430336b7eabc9b6ee26f7 not found: ID does not exist" Jul 17 07:53:11.602046 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:11.602020 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl"] Jul 17 07:53:11.606468 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:11.606447 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-f88ba1e3-openshift-default-7bfb4bf647-9w7hl"] Jul 17 07:53:12.000807 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:12.000775 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3825789-59b7-4e9c-86ff-b207b8335ac4" path="/var/lib/kubelet/pods/e3825789-59b7-4e9c-86ff-b207b8335ac4/volumes" Jul 17 07:53:27.252498 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:27.252460 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2"] Jul 17 07:53:27.252912 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:27.252858 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3825789-59b7-4e9c-86ff-b207b8335ac4" containerName="istio-proxy" Jul 17 07:53:27.252912 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:27.252869 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3825789-59b7-4e9c-86ff-b207b8335ac4" containerName="istio-proxy" Jul 17 07:53:27.252912 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:27.252879 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3825789-59b7-4e9c-86ff-b207b8335ac4" containerName="istio-proxy" Jul 17 07:53:27.252912 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:27.252885 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3825789-59b7-4e9c-86ff-b207b8335ac4" containerName="istio-proxy" Jul 17 07:53:27.253067 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:27.252942 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3825789-59b7-4e9c-86ff-b207b8335ac4" containerName="istio-proxy" Jul 17 07:53:27.253067 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:27.252950 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3825789-59b7-4e9c-86ff-b207b8335ac4" containerName="istio-proxy" Jul 17 07:53:27.256137 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:27.256115 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2" Jul 17 07:53:27.258673 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:27.258653 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-e8d5b32b-openshift-default-dockercfg-tsnwm\"" Jul 17 07:53:27.279462 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:27.279438 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2"] Jul 17 07:53:27.324951 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:27.324917 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/018fcc3a-20dc-4758-80bd-950354a2022e-istio-envoy\") pod \"e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2\" (UID: \"018fcc3a-20dc-4758-80bd-950354a2022e\") " pod="openshift-ingress/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2" Jul 17 07:53:27.325099 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:27.324965 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/018fcc3a-20dc-4758-80bd-950354a2022e-istiod-ca-cert\") pod \"e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2\" (UID: \"018fcc3a-20dc-4758-80bd-950354a2022e\") " pod="openshift-ingress/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2" Jul 17 07:53:27.325099 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:27.325034 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/018fcc3a-20dc-4758-80bd-950354a2022e-workload-socket\") pod \"e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2\" (UID: \"018fcc3a-20dc-4758-80bd-950354a2022e\") " pod="openshift-ingress/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2" Jul 17 07:53:27.325188 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:27.325106 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7n7k\" (UniqueName: \"kubernetes.io/projected/018fcc3a-20dc-4758-80bd-950354a2022e-kube-api-access-r7n7k\") pod \"e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2\" (UID: \"018fcc3a-20dc-4758-80bd-950354a2022e\") " pod="openshift-ingress/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2" Jul 17 07:53:27.325188 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:27.325155 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/018fcc3a-20dc-4758-80bd-950354a2022e-workload-certs\") pod \"e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2\" (UID: \"018fcc3a-20dc-4758-80bd-950354a2022e\") " pod="openshift-ingress/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2" Jul 17 07:53:27.325188 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:27.325174 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/018fcc3a-20dc-4758-80bd-950354a2022e-istio-data\") pod \"e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2\" (UID: \"018fcc3a-20dc-4758-80bd-950354a2022e\") " pod="openshift-ingress/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2" Jul 17 07:53:27.325332 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:27.325219 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/018fcc3a-20dc-4758-80bd-950354a2022e-istio-podinfo\") pod \"e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2\" (UID: \"018fcc3a-20dc-4758-80bd-950354a2022e\") " pod="openshift-ingress/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2" Jul 17 07:53:27.325332 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:27.325257 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/018fcc3a-20dc-4758-80bd-950354a2022e-credential-socket\") pod \"e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2\" (UID: \"018fcc3a-20dc-4758-80bd-950354a2022e\") " pod="openshift-ingress/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2" Jul 17 07:53:27.325332 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:27.325304 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/018fcc3a-20dc-4758-80bd-950354a2022e-istio-token\") pod \"e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2\" (UID: \"018fcc3a-20dc-4758-80bd-950354a2022e\") " pod="openshift-ingress/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2" Jul 17 07:53:27.426750 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:27.426712 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/018fcc3a-20dc-4758-80bd-950354a2022e-istiod-ca-cert\") pod \"e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2\" (UID: \"018fcc3a-20dc-4758-80bd-950354a2022e\") " pod="openshift-ingress/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2" Jul 17 07:53:27.426902 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:27.426766 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/018fcc3a-20dc-4758-80bd-950354a2022e-workload-socket\") pod \"e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2\" (UID: \"018fcc3a-20dc-4758-80bd-950354a2022e\") " pod="openshift-ingress/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2" Jul 17 07:53:27.426902 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:27.426812 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7n7k\" (UniqueName: \"kubernetes.io/projected/018fcc3a-20dc-4758-80bd-950354a2022e-kube-api-access-r7n7k\") pod \"e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2\" (UID: \"018fcc3a-20dc-4758-80bd-950354a2022e\") " pod="openshift-ingress/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2" Jul 17 07:53:27.426902 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:27.426857 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/018fcc3a-20dc-4758-80bd-950354a2022e-workload-certs\") pod \"e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2\" (UID: \"018fcc3a-20dc-4758-80bd-950354a2022e\") " pod="openshift-ingress/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2" Jul 17 07:53:27.426902 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:27.426882 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/018fcc3a-20dc-4758-80bd-950354a2022e-istio-data\") pod \"e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2\" (UID: \"018fcc3a-20dc-4758-80bd-950354a2022e\") " pod="openshift-ingress/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2" Jul 17 07:53:27.427123 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:27.426910 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/018fcc3a-20dc-4758-80bd-950354a2022e-istio-podinfo\") pod \"e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2\" (UID: \"018fcc3a-20dc-4758-80bd-950354a2022e\") " pod="openshift-ingress/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2" Jul 17 07:53:27.427123 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:27.426952 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/018fcc3a-20dc-4758-80bd-950354a2022e-credential-socket\") pod \"e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2\" (UID: \"018fcc3a-20dc-4758-80bd-950354a2022e\") " pod="openshift-ingress/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2" Jul 17 07:53:27.427123 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:27.426977 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/018fcc3a-20dc-4758-80bd-950354a2022e-istio-token\") pod \"e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2\" (UID: \"018fcc3a-20dc-4758-80bd-950354a2022e\") " pod="openshift-ingress/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2" Jul 17 07:53:27.427123 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:27.427021 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/018fcc3a-20dc-4758-80bd-950354a2022e-istio-envoy\") pod \"e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2\" (UID: \"018fcc3a-20dc-4758-80bd-950354a2022e\") " pod="openshift-ingress/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2" Jul 17 07:53:27.427374 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:27.427240 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/018fcc3a-20dc-4758-80bd-950354a2022e-workload-socket\") pod \"e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2\" (UID: \"018fcc3a-20dc-4758-80bd-950354a2022e\") " pod="openshift-ingress/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2" Jul 17 07:53:27.427374 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:27.427319 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/018fcc3a-20dc-4758-80bd-950354a2022e-istio-data\") pod \"e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2\" (UID: \"018fcc3a-20dc-4758-80bd-950354a2022e\") " pod="openshift-ingress/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2" Jul 17 07:53:27.427374 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:27.427352 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/018fcc3a-20dc-4758-80bd-950354a2022e-workload-certs\") pod \"e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2\" (UID: \"018fcc3a-20dc-4758-80bd-950354a2022e\") " pod="openshift-ingress/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2" Jul 17 07:53:27.427564 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:27.427543 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/018fcc3a-20dc-4758-80bd-950354a2022e-credential-socket\") pod \"e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2\" (UID: \"018fcc3a-20dc-4758-80bd-950354a2022e\") " pod="openshift-ingress/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2" Jul 17 07:53:27.427619 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:27.427556 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/018fcc3a-20dc-4758-80bd-950354a2022e-istiod-ca-cert\") pod \"e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2\" (UID: \"018fcc3a-20dc-4758-80bd-950354a2022e\") " pod="openshift-ingress/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2" Jul 17 07:53:27.429231 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:27.429210 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/018fcc3a-20dc-4758-80bd-950354a2022e-istio-envoy\") pod \"e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2\" (UID: \"018fcc3a-20dc-4758-80bd-950354a2022e\") " pod="openshift-ingress/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2" Jul 17 07:53:27.429457 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:27.429440 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/018fcc3a-20dc-4758-80bd-950354a2022e-istio-podinfo\") pod \"e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2\" (UID: \"018fcc3a-20dc-4758-80bd-950354a2022e\") " pod="openshift-ingress/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2" Jul 17 07:53:27.439074 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:27.439052 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7n7k\" (UniqueName: \"kubernetes.io/projected/018fcc3a-20dc-4758-80bd-950354a2022e-kube-api-access-r7n7k\") pod \"e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2\" (UID: \"018fcc3a-20dc-4758-80bd-950354a2022e\") " pod="openshift-ingress/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2" Jul 17 07:53:27.439351 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:27.439334 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/018fcc3a-20dc-4758-80bd-950354a2022e-istio-token\") pod \"e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2\" (UID: \"018fcc3a-20dc-4758-80bd-950354a2022e\") " pod="openshift-ingress/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2" Jul 17 07:53:27.566227 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:27.566153 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2" Jul 17 07:53:27.714565 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:27.714536 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2"] Jul 17 07:53:27.716418 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:53:27.716390 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod018fcc3a_20dc_4758_80bd_950354a2022e.slice/crio-8e3f7beada92da45893249cbc6f93d4d11df18ae98de4b17f58305eb09a6aa73 WatchSource:0}: Error finding container 8e3f7beada92da45893249cbc6f93d4d11df18ae98de4b17f58305eb09a6aa73: Status 404 returned error can't find the container with id 8e3f7beada92da45893249cbc6f93d4d11df18ae98de4b17f58305eb09a6aa73 Jul 17 07:53:27.718538 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:27.718502 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:53:27.718639 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:27.718585 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:53:27.718707 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:27.718638 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:53:28.672983 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:28.672891 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2" event={"ID":"018fcc3a-20dc-4758-80bd-950354a2022e","Type":"ContainerStarted","Data":"39737f8b9acacc7675382a3936e72acc2bf8231123bd1cbec7b6406c21d3e962"} Jul 17 07:53:28.672983 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:28.672960 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2" event={"ID":"018fcc3a-20dc-4758-80bd-950354a2022e","Type":"ContainerStarted","Data":"8e3f7beada92da45893249cbc6f93d4d11df18ae98de4b17f58305eb09a6aa73"} Jul 17 07:53:28.696457 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:28.696402 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2" podStartSLOduration=1.6963863240000001 podStartE2EDuration="1.696386324s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 07:53:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:53:28.694734851 +0000 UTC m=+2513.179423825" watchObservedRunningTime="2026-07-17 07:53:28.696386324 +0000 UTC m=+2513.181075298" Jul 17 07:53:29.567144 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:29.567112 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2" Jul 17 07:53:29.571888 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:29.571864 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2" Jul 17 07:53:29.678860 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:29.678825 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2" Jul 17 07:53:29.679706 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:29.679684 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2" Jul 17 07:53:45.224231 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:45.223334 2568 patch_prober.go:28] interesting pod/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": read tcp 10.132.0.2:42958->10.132.0.63:15021: read: connection reset by peer" start-of-body= Jul 17 07:53:45.224231 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:45.223405 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2" podUID="018fcc3a-20dc-4758-80bd-950354a2022e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": read tcp 10.132.0.2:42958->10.132.0.63:15021: read: connection reset by peer" Jul 17 07:53:45.771899 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:45.771866 2568 generic.go:356] "Generic (PLEG): container finished" podID="018fcc3a-20dc-4758-80bd-950354a2022e" containerID="39737f8b9acacc7675382a3936e72acc2bf8231123bd1cbec7b6406c21d3e962" exitCode=0 Jul 17 07:53:45.772062 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:45.771904 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2" event={"ID":"018fcc3a-20dc-4758-80bd-950354a2022e","Type":"ContainerDied","Data":"39737f8b9acacc7675382a3936e72acc2bf8231123bd1cbec7b6406c21d3e962"} Jul 17 07:53:45.772457 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:45.772440 2568 scope.go:117] "RemoveContainer" containerID="39737f8b9acacc7675382a3936e72acc2bf8231123bd1cbec7b6406c21d3e962" Jul 17 07:53:45.773571 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:45.773544 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:53:45.773640 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:45.773623 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:53:45.773678 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:45.773654 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:53:46.783682 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:46.783634 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2" event={"ID":"018fcc3a-20dc-4758-80bd-950354a2022e","Type":"ContainerStarted","Data":"d7a1d72b10f728d014981f5bc6483a5f0a3d8bd6435b3457ef61ec432f100ab4"} Jul 17 07:53:47.566351 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:47.566279 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2" Jul 17 07:53:47.566589 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:47.566559 2568 patch_prober.go:28] interesting pod/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 17 07:53:47.566812 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:47.566785 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2" podUID="018fcc3a-20dc-4758-80bd-950354a2022e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 17 07:53:48.567499 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:48.567455 2568 patch_prober.go:28] interesting pod/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 17 07:53:48.568007 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:48.567528 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2" podUID="018fcc3a-20dc-4758-80bd-950354a2022e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 17 07:53:49.567494 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:49.567456 2568 patch_prober.go:28] interesting pod/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 17 07:53:49.567705 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:49.567531 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2" podUID="018fcc3a-20dc-4758-80bd-950354a2022e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 17 07:53:50.569735 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:50.569698 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2" Jul 17 07:53:50.570221 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:50.569976 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2" Jul 17 07:53:50.570734 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:53:50.570717 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2" Jul 17 07:55:41.628241 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:55:41.628198 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2"] Jul 17 07:55:41.628890 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:55:41.628477 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2" podUID="018fcc3a-20dc-4758-80bd-950354a2022e" containerName="istio-proxy" containerID="cri-o://d7a1d72b10f728d014981f5bc6483a5f0a3d8bd6435b3457ef61ec432f100ab4" gracePeriod=30 Jul 17 07:55:46.896748 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:55:46.896689 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2" Jul 17 07:55:46.994107 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:55:46.994077 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/018fcc3a-20dc-4758-80bd-950354a2022e-istiod-ca-cert\") pod \"018fcc3a-20dc-4758-80bd-950354a2022e\" (UID: \"018fcc3a-20dc-4758-80bd-950354a2022e\") " Jul 17 07:55:46.994297 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:55:46.994126 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/018fcc3a-20dc-4758-80bd-950354a2022e-workload-socket\") pod \"018fcc3a-20dc-4758-80bd-950354a2022e\" (UID: \"018fcc3a-20dc-4758-80bd-950354a2022e\") " Jul 17 07:55:46.994297 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:55:46.994160 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r7n7k\" (UniqueName: \"kubernetes.io/projected/018fcc3a-20dc-4758-80bd-950354a2022e-kube-api-access-r7n7k\") pod \"018fcc3a-20dc-4758-80bd-950354a2022e\" (UID: \"018fcc3a-20dc-4758-80bd-950354a2022e\") " Jul 17 07:55:46.994435 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:55:46.994308 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/018fcc3a-20dc-4758-80bd-950354a2022e-istio-podinfo\") pod \"018fcc3a-20dc-4758-80bd-950354a2022e\" (UID: \"018fcc3a-20dc-4758-80bd-950354a2022e\") " Jul 17 07:55:46.994435 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:55:46.994350 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/018fcc3a-20dc-4758-80bd-950354a2022e-workload-certs\") pod \"018fcc3a-20dc-4758-80bd-950354a2022e\" (UID: \"018fcc3a-20dc-4758-80bd-950354a2022e\") " Jul 17 07:55:46.994435 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:55:46.994385 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/018fcc3a-20dc-4758-80bd-950354a2022e-istio-envoy\") pod \"018fcc3a-20dc-4758-80bd-950354a2022e\" (UID: \"018fcc3a-20dc-4758-80bd-950354a2022e\") " Jul 17 07:55:46.994435 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:55:46.994401 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/018fcc3a-20dc-4758-80bd-950354a2022e-istio-data\") pod \"018fcc3a-20dc-4758-80bd-950354a2022e\" (UID: \"018fcc3a-20dc-4758-80bd-950354a2022e\") " Jul 17 07:55:46.994435 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:55:46.994419 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/018fcc3a-20dc-4758-80bd-950354a2022e-istio-token\") pod \"018fcc3a-20dc-4758-80bd-950354a2022e\" (UID: \"018fcc3a-20dc-4758-80bd-950354a2022e\") " Jul 17 07:55:46.994695 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:55:46.994437 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/018fcc3a-20dc-4758-80bd-950354a2022e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "018fcc3a-20dc-4758-80bd-950354a2022e" (UID: "018fcc3a-20dc-4758-80bd-950354a2022e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:55:46.994695 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:55:46.994448 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/018fcc3a-20dc-4758-80bd-950354a2022e-credential-socket\") pod \"018fcc3a-20dc-4758-80bd-950354a2022e\" (UID: \"018fcc3a-20dc-4758-80bd-950354a2022e\") " Jul 17 07:55:46.994695 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:55:46.994489 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/018fcc3a-20dc-4758-80bd-950354a2022e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "018fcc3a-20dc-4758-80bd-950354a2022e" (UID: "018fcc3a-20dc-4758-80bd-950354a2022e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 07:55:46.994695 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:55:46.994613 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/018fcc3a-20dc-4758-80bd-950354a2022e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "018fcc3a-20dc-4758-80bd-950354a2022e" (UID: "018fcc3a-20dc-4758-80bd-950354a2022e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:55:46.994695 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:55:46.994631 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/018fcc3a-20dc-4758-80bd-950354a2022e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "018fcc3a-20dc-4758-80bd-950354a2022e" (UID: "018fcc3a-20dc-4758-80bd-950354a2022e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:55:46.994982 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:55:46.994899 2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/018fcc3a-20dc-4758-80bd-950354a2022e-istiod-ca-cert\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:55:46.994982 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:55:46.994920 2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/018fcc3a-20dc-4758-80bd-950354a2022e-workload-socket\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:55:46.994982 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:55:46.994934 2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/018fcc3a-20dc-4758-80bd-950354a2022e-workload-certs\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:55:46.994982 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:55:46.994947 2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/018fcc3a-20dc-4758-80bd-950354a2022e-credential-socket\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:55:46.995554 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:55:46.995525 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/018fcc3a-20dc-4758-80bd-950354a2022e-istio-data" (OuterVolumeSpecName: "istio-data") pod "018fcc3a-20dc-4758-80bd-950354a2022e" (UID: "018fcc3a-20dc-4758-80bd-950354a2022e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:55:46.996555 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:55:46.996533 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/018fcc3a-20dc-4758-80bd-950354a2022e-kube-api-access-r7n7k" (OuterVolumeSpecName: "kube-api-access-r7n7k") pod "018fcc3a-20dc-4758-80bd-950354a2022e" (UID: "018fcc3a-20dc-4758-80bd-950354a2022e"). InnerVolumeSpecName "kube-api-access-r7n7k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 07:55:46.996745 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:55:46.996727 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/018fcc3a-20dc-4758-80bd-950354a2022e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "018fcc3a-20dc-4758-80bd-950354a2022e" (UID: "018fcc3a-20dc-4758-80bd-950354a2022e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 17 07:55:46.997051 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:55:46.997030 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/018fcc3a-20dc-4758-80bd-950354a2022e-istio-token" (OuterVolumeSpecName: "istio-token") pod "018fcc3a-20dc-4758-80bd-950354a2022e" (UID: "018fcc3a-20dc-4758-80bd-950354a2022e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 07:55:46.997151 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:55:46.997034 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/018fcc3a-20dc-4758-80bd-950354a2022e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "018fcc3a-20dc-4758-80bd-950354a2022e" (UID: "018fcc3a-20dc-4758-80bd-950354a2022e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:55:47.095865 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:55:47.095831 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r7n7k\" (UniqueName: \"kubernetes.io/projected/018fcc3a-20dc-4758-80bd-950354a2022e-kube-api-access-r7n7k\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:55:47.095865 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:55:47.095861 2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/018fcc3a-20dc-4758-80bd-950354a2022e-istio-podinfo\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:55:47.095865 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:55:47.095871 2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/018fcc3a-20dc-4758-80bd-950354a2022e-istio-envoy\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:55:47.096100 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:55:47.095880 2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/018fcc3a-20dc-4758-80bd-950354a2022e-istio-data\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:55:47.096100 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:55:47.095887 2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/018fcc3a-20dc-4758-80bd-950354a2022e-istio-token\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:55:47.441404 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:55:47.441368 2568 generic.go:356] "Generic (PLEG): container finished" podID="018fcc3a-20dc-4758-80bd-950354a2022e" containerID="d7a1d72b10f728d014981f5bc6483a5f0a3d8bd6435b3457ef61ec432f100ab4" exitCode=0 Jul 17 07:55:47.441610 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:55:47.441423 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2" Jul 17 07:55:47.441610 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:55:47.441454 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2" event={"ID":"018fcc3a-20dc-4758-80bd-950354a2022e","Type":"ContainerDied","Data":"d7a1d72b10f728d014981f5bc6483a5f0a3d8bd6435b3457ef61ec432f100ab4"} Jul 17 07:55:47.441610 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:55:47.441518 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2" event={"ID":"018fcc3a-20dc-4758-80bd-950354a2022e","Type":"ContainerDied","Data":"8e3f7beada92da45893249cbc6f93d4d11df18ae98de4b17f58305eb09a6aa73"} Jul 17 07:55:47.441610 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:55:47.441549 2568 scope.go:117] "RemoveContainer" containerID="d7a1d72b10f728d014981f5bc6483a5f0a3d8bd6435b3457ef61ec432f100ab4" Jul 17 07:55:47.451121 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:55:47.451106 2568 scope.go:117] "RemoveContainer" containerID="39737f8b9acacc7675382a3936e72acc2bf8231123bd1cbec7b6406c21d3e962" Jul 17 07:55:47.459464 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:55:47.459408 2568 scope.go:117] "RemoveContainer" containerID="d7a1d72b10f728d014981f5bc6483a5f0a3d8bd6435b3457ef61ec432f100ab4" Jul 17 07:55:47.459669 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:55:47.459650 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7a1d72b10f728d014981f5bc6483a5f0a3d8bd6435b3457ef61ec432f100ab4\": container with ID starting with d7a1d72b10f728d014981f5bc6483a5f0a3d8bd6435b3457ef61ec432f100ab4 not found: ID does not exist" containerID="d7a1d72b10f728d014981f5bc6483a5f0a3d8bd6435b3457ef61ec432f100ab4" Jul 17 07:55:47.459742 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:55:47.459681 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7a1d72b10f728d014981f5bc6483a5f0a3d8bd6435b3457ef61ec432f100ab4"} err="failed to get container status \"d7a1d72b10f728d014981f5bc6483a5f0a3d8bd6435b3457ef61ec432f100ab4\": rpc error: code = NotFound desc = could not find container \"d7a1d72b10f728d014981f5bc6483a5f0a3d8bd6435b3457ef61ec432f100ab4\": container with ID starting with d7a1d72b10f728d014981f5bc6483a5f0a3d8bd6435b3457ef61ec432f100ab4 not found: ID does not exist" Jul 17 07:55:47.459742 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:55:47.459705 2568 scope.go:117] "RemoveContainer" containerID="39737f8b9acacc7675382a3936e72acc2bf8231123bd1cbec7b6406c21d3e962" Jul 17 07:55:47.459953 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:55:47.459935 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39737f8b9acacc7675382a3936e72acc2bf8231123bd1cbec7b6406c21d3e962\": container with ID starting with 39737f8b9acacc7675382a3936e72acc2bf8231123bd1cbec7b6406c21d3e962 not found: ID does not exist" containerID="39737f8b9acacc7675382a3936e72acc2bf8231123bd1cbec7b6406c21d3e962" Jul 17 07:55:47.459993 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:55:47.459959 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39737f8b9acacc7675382a3936e72acc2bf8231123bd1cbec7b6406c21d3e962"} err="failed to get container status \"39737f8b9acacc7675382a3936e72acc2bf8231123bd1cbec7b6406c21d3e962\": rpc error: code = NotFound desc = could not find container \"39737f8b9acacc7675382a3936e72acc2bf8231123bd1cbec7b6406c21d3e962\": container with ID starting with 39737f8b9acacc7675382a3936e72acc2bf8231123bd1cbec7b6406c21d3e962 not found: ID does not exist" Jul 17 07:55:47.471657 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:55:47.471637 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2"] Jul 17 07:55:47.483245 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:55:47.483220 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-e8d5b32b-openshift-default-6b6c864dbc-lwtk2"] Jul 17 07:55:47.999857 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:55:47.999825 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="018fcc3a-20dc-4758-80bd-950354a2022e" path="/var/lib/kubelet/pods/018fcc3a-20dc-4758-80bd-950354a2022e/volumes" Jul 17 07:56:36.249474 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:36.249390 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-xr6pg_2ce00a54-15e9-42b7-b3a8-9e58d1101558/cluster-monitoring-operator/0.log" Jul 17 07:56:36.254155 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:36.254119 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-xpxv5_27538d1e-992b-4651-a2d2-7a7ecf442806/console-operator/1.log" Jul 17 07:56:36.257839 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:36.257813 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cpwz4_1c44a3ae-4c83-4203-8b0b-116d6618accf/ovn-acl-logging/0.log" Jul 17 07:56:36.261418 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:36.261396 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-xr6pg_2ce00a54-15e9-42b7-b3a8-9e58d1101558/cluster-monitoring-operator/0.log" Jul 17 07:56:36.270066 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:36.270046 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-xpxv5_27538d1e-992b-4651-a2d2-7a7ecf442806/console-operator/1.log" Jul 17 07:56:36.274076 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:36.274060 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cpwz4_1c44a3ae-4c83-4203-8b0b-116d6618accf/ovn-acl-logging/0.log" Jul 17 07:56:38.674416 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:38.674379 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw"] Jul 17 07:56:38.674879 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:38.674861 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="018fcc3a-20dc-4758-80bd-950354a2022e" containerName="istio-proxy" Jul 17 07:56:38.674947 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:38.674882 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="018fcc3a-20dc-4758-80bd-950354a2022e" containerName="istio-proxy" Jul 17 07:56:38.675009 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:38.674972 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="018fcc3a-20dc-4758-80bd-950354a2022e" containerName="istio-proxy" Jul 17 07:56:38.675009 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:38.674981 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="018fcc3a-20dc-4758-80bd-950354a2022e" containerName="istio-proxy" Jul 17 07:56:38.679277 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:38.679259 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw" Jul 17 07:56:38.681811 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:38.681791 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-1f268b-openshift-default-dockercfg-hdzc9\"" Jul 17 07:56:38.691032 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:38.691010 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw"] Jul 17 07:56:38.756105 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:38.756060 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f1bb1bb-0203-46c6-9089-429977417047-workload-certs\") pod \"e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw\" (UID: \"5f1bb1bb-0203-46c6-9089-429977417047\") " pod="openshift-ingress/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw" Jul 17 07:56:38.756275 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:38.756239 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hplwl\" (UniqueName: \"kubernetes.io/projected/5f1bb1bb-0203-46c6-9089-429977417047-kube-api-access-hplwl\") pod \"e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw\" (UID: \"5f1bb1bb-0203-46c6-9089-429977417047\") " pod="openshift-ingress/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw" Jul 17 07:56:38.756361 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:38.756315 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f1bb1bb-0203-46c6-9089-429977417047-workload-socket\") pod \"e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw\" (UID: \"5f1bb1bb-0203-46c6-9089-429977417047\") " pod="openshift-ingress/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw" Jul 17 07:56:38.756488 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:38.756364 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f1bb1bb-0203-46c6-9089-429977417047-credential-socket\") pod \"e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw\" (UID: \"5f1bb1bb-0203-46c6-9089-429977417047\") " pod="openshift-ingress/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw" Jul 17 07:56:38.756540 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:38.756478 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f1bb1bb-0203-46c6-9089-429977417047-istio-data\") pod \"e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw\" (UID: \"5f1bb1bb-0203-46c6-9089-429977417047\") " pod="openshift-ingress/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw" Jul 17 07:56:38.756586 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:38.756543 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f1bb1bb-0203-46c6-9089-429977417047-istio-token\") pod \"e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw\" (UID: \"5f1bb1bb-0203-46c6-9089-429977417047\") " pod="openshift-ingress/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw" Jul 17 07:56:38.756631 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:38.756587 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f1bb1bb-0203-46c6-9089-429977417047-istiod-ca-cert\") pod \"e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw\" (UID: \"5f1bb1bb-0203-46c6-9089-429977417047\") " pod="openshift-ingress/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw" Jul 17 07:56:38.756631 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:38.756615 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f1bb1bb-0203-46c6-9089-429977417047-istio-podinfo\") pod \"e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw\" (UID: \"5f1bb1bb-0203-46c6-9089-429977417047\") " pod="openshift-ingress/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw" Jul 17 07:56:38.756737 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:38.756642 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f1bb1bb-0203-46c6-9089-429977417047-istio-envoy\") pod \"e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw\" (UID: \"5f1bb1bb-0203-46c6-9089-429977417047\") " pod="openshift-ingress/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw" Jul 17 07:56:38.857389 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:38.857355 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hplwl\" (UniqueName: \"kubernetes.io/projected/5f1bb1bb-0203-46c6-9089-429977417047-kube-api-access-hplwl\") pod \"e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw\" (UID: \"5f1bb1bb-0203-46c6-9089-429977417047\") " pod="openshift-ingress/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw" Jul 17 07:56:38.857389 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:38.857392 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f1bb1bb-0203-46c6-9089-429977417047-workload-socket\") pod \"e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw\" (UID: \"5f1bb1bb-0203-46c6-9089-429977417047\") " pod="openshift-ingress/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw" Jul 17 07:56:38.857642 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:38.857415 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f1bb1bb-0203-46c6-9089-429977417047-credential-socket\") pod \"e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw\" (UID: \"5f1bb1bb-0203-46c6-9089-429977417047\") " pod="openshift-ingress/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw" Jul 17 07:56:38.857642 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:38.857456 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f1bb1bb-0203-46c6-9089-429977417047-istio-data\") pod \"e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw\" (UID: \"5f1bb1bb-0203-46c6-9089-429977417047\") " pod="openshift-ingress/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw" Jul 17 07:56:38.857642 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:38.857492 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f1bb1bb-0203-46c6-9089-429977417047-istio-token\") pod \"e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw\" (UID: \"5f1bb1bb-0203-46c6-9089-429977417047\") " pod="openshift-ingress/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw" Jul 17 07:56:38.857642 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:38.857524 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f1bb1bb-0203-46c6-9089-429977417047-istiod-ca-cert\") pod \"e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw\" (UID: \"5f1bb1bb-0203-46c6-9089-429977417047\") " pod="openshift-ingress/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw" Jul 17 07:56:38.857642 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:38.857547 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f1bb1bb-0203-46c6-9089-429977417047-istio-podinfo\") pod \"e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw\" (UID: \"5f1bb1bb-0203-46c6-9089-429977417047\") " pod="openshift-ingress/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw" Jul 17 07:56:38.857642 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:38.857574 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f1bb1bb-0203-46c6-9089-429977417047-istio-envoy\") pod \"e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw\" (UID: \"5f1bb1bb-0203-46c6-9089-429977417047\") " pod="openshift-ingress/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw" Jul 17 07:56:38.857642 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:38.857616 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f1bb1bb-0203-46c6-9089-429977417047-workload-certs\") pod \"e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw\" (UID: \"5f1bb1bb-0203-46c6-9089-429977417047\") " pod="openshift-ingress/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw" Jul 17 07:56:38.858112 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:38.857963 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f1bb1bb-0203-46c6-9089-429977417047-workload-socket\") pod \"e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw\" (UID: \"5f1bb1bb-0203-46c6-9089-429977417047\") " pod="openshift-ingress/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw" Jul 17 07:56:38.858112 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:38.858002 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f1bb1bb-0203-46c6-9089-429977417047-istio-data\") pod \"e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw\" (UID: \"5f1bb1bb-0203-46c6-9089-429977417047\") " pod="openshift-ingress/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw" Jul 17 07:56:38.858229 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:38.858124 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f1bb1bb-0203-46c6-9089-429977417047-credential-socket\") pod \"e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw\" (UID: \"5f1bb1bb-0203-46c6-9089-429977417047\") " pod="openshift-ingress/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw" Jul 17 07:56:38.858229 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:38.858141 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f1bb1bb-0203-46c6-9089-429977417047-workload-certs\") pod \"e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw\" (UID: \"5f1bb1bb-0203-46c6-9089-429977417047\") " pod="openshift-ingress/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw" Jul 17 07:56:38.858443 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:38.858416 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f1bb1bb-0203-46c6-9089-429977417047-istiod-ca-cert\") pod \"e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw\" (UID: \"5f1bb1bb-0203-46c6-9089-429977417047\") " pod="openshift-ingress/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw" Jul 17 07:56:38.860411 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:38.860384 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f1bb1bb-0203-46c6-9089-429977417047-istio-envoy\") pod \"e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw\" (UID: \"5f1bb1bb-0203-46c6-9089-429977417047\") " pod="openshift-ingress/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw" Jul 17 07:56:38.860644 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:38.860627 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f1bb1bb-0203-46c6-9089-429977417047-istio-podinfo\") pod \"e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw\" (UID: \"5f1bb1bb-0203-46c6-9089-429977417047\") " pod="openshift-ingress/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw" Jul 17 07:56:38.866160 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:38.866138 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f1bb1bb-0203-46c6-9089-429977417047-istio-token\") pod \"e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw\" (UID: \"5f1bb1bb-0203-46c6-9089-429977417047\") " pod="openshift-ingress/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw" Jul 17 07:56:38.866464 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:38.866447 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hplwl\" (UniqueName: \"kubernetes.io/projected/5f1bb1bb-0203-46c6-9089-429977417047-kube-api-access-hplwl\") pod \"e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw\" (UID: \"5f1bb1bb-0203-46c6-9089-429977417047\") " pod="openshift-ingress/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw" Jul 17 07:56:38.990303 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:38.990208 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw" Jul 17 07:56:39.134973 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:39.134948 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw"] Jul 17 07:56:39.136394 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:56:39.136368 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f1bb1bb_0203_46c6_9089_429977417047.slice/crio-7c7b2c325d1d77cae46838b92097970e49cf8ce9d994eec35680e07ca8d5b1b7 WatchSource:0}: Error finding container 7c7b2c325d1d77cae46838b92097970e49cf8ce9d994eec35680e07ca8d5b1b7: Status 404 returned error can't find the container with id 7c7b2c325d1d77cae46838b92097970e49cf8ce9d994eec35680e07ca8d5b1b7 Jul 17 07:56:39.138590 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:39.138557 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:56:39.138669 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:39.138624 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:56:39.138669 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:39.138653 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:56:39.720714 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:39.720678 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw" event={"ID":"5f1bb1bb-0203-46c6-9089-429977417047","Type":"ContainerStarted","Data":"c669078c08ba82d0c39af3230464ac33a6e03f705b535d154ce4c25d21e51846"} Jul 17 07:56:39.721106 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:39.720758 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw" event={"ID":"5f1bb1bb-0203-46c6-9089-429977417047","Type":"ContainerStarted","Data":"7c7b2c325d1d77cae46838b92097970e49cf8ce9d994eec35680e07ca8d5b1b7"} Jul 17 07:56:39.744435 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:39.744378 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw" podStartSLOduration=1.744361925 podStartE2EDuration="1.744361925s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 07:56:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:56:39.743202278 +0000 UTC m=+2704.227891251" watchObservedRunningTime="2026-07-17 07:56:39.744361925 +0000 UTC m=+2704.229050947" Jul 17 07:56:39.990733 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:39.990651 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw" Jul 17 07:56:40.995720 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:40.995685 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw" Jul 17 07:56:41.732317 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:41.732272 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw" Jul 17 07:56:41.733320 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:41.733298 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw" Jul 17 07:56:46.989132 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:46.989098 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm"] Jul 17 07:56:46.989543 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:46.989522 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="018fcc3a-20dc-4758-80bd-950354a2022e" containerName="istio-proxy" Jul 17 07:56:46.989543 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:46.989533 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="018fcc3a-20dc-4758-80bd-950354a2022e" containerName="istio-proxy" Jul 17 07:56:46.992750 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:46.992723 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm" Jul 17 07:56:46.996566 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:46.996545 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-cab91c-openshift-default-dockercfg-p4mvg\"" Jul 17 07:56:47.014425 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:47.014402 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm"] Jul 17 07:56:47.133715 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:47.133674 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95414144-5e2c-4b7c-bbba-d72ae890b0c5-istio-podinfo\") pod \"e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm\" (UID: \"95414144-5e2c-4b7c-bbba-d72ae890b0c5\") " pod="openshift-ingress/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm" Jul 17 07:56:47.133887 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:47.133755 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95414144-5e2c-4b7c-bbba-d72ae890b0c5-workload-certs\") pod \"e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm\" (UID: \"95414144-5e2c-4b7c-bbba-d72ae890b0c5\") " pod="openshift-ingress/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm" Jul 17 07:56:47.133887 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:47.133811 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95414144-5e2c-4b7c-bbba-d72ae890b0c5-workload-socket\") pod \"e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm\" (UID: \"95414144-5e2c-4b7c-bbba-d72ae890b0c5\") " pod="openshift-ingress/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm" Jul 17 07:56:47.133887 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:47.133863 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95414144-5e2c-4b7c-bbba-d72ae890b0c5-credential-socket\") pod \"e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm\" (UID: \"95414144-5e2c-4b7c-bbba-d72ae890b0c5\") " pod="openshift-ingress/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm" Jul 17 07:56:47.133998 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:47.133893 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95414144-5e2c-4b7c-bbba-d72ae890b0c5-istio-data\") pod \"e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm\" (UID: \"95414144-5e2c-4b7c-bbba-d72ae890b0c5\") " pod="openshift-ingress/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm" Jul 17 07:56:47.133998 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:47.133913 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95414144-5e2c-4b7c-bbba-d72ae890b0c5-istio-token\") pod \"e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm\" (UID: \"95414144-5e2c-4b7c-bbba-d72ae890b0c5\") " pod="openshift-ingress/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm" Jul 17 07:56:47.133998 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:47.133976 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95414144-5e2c-4b7c-bbba-d72ae890b0c5-istiod-ca-cert\") pod \"e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm\" (UID: \"95414144-5e2c-4b7c-bbba-d72ae890b0c5\") " pod="openshift-ingress/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm" Jul 17 07:56:47.134097 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:47.134002 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95414144-5e2c-4b7c-bbba-d72ae890b0c5-istio-envoy\") pod \"e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm\" (UID: \"95414144-5e2c-4b7c-bbba-d72ae890b0c5\") " pod="openshift-ingress/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm" Jul 17 07:56:47.134097 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:47.134066 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7sv9q\" (UniqueName: \"kubernetes.io/projected/95414144-5e2c-4b7c-bbba-d72ae890b0c5-kube-api-access-7sv9q\") pod \"e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm\" (UID: \"95414144-5e2c-4b7c-bbba-d72ae890b0c5\") " pod="openshift-ingress/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm" Jul 17 07:56:47.235505 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:47.235467 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95414144-5e2c-4b7c-bbba-d72ae890b0c5-istiod-ca-cert\") pod \"e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm\" (UID: \"95414144-5e2c-4b7c-bbba-d72ae890b0c5\") " pod="openshift-ingress/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm" Jul 17 07:56:47.235505 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:47.235506 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95414144-5e2c-4b7c-bbba-d72ae890b0c5-istio-envoy\") pod \"e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm\" (UID: \"95414144-5e2c-4b7c-bbba-d72ae890b0c5\") " pod="openshift-ingress/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm" Jul 17 07:56:47.235748 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:47.235540 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7sv9q\" (UniqueName: \"kubernetes.io/projected/95414144-5e2c-4b7c-bbba-d72ae890b0c5-kube-api-access-7sv9q\") pod \"e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm\" (UID: \"95414144-5e2c-4b7c-bbba-d72ae890b0c5\") " pod="openshift-ingress/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm" Jul 17 07:56:47.235748 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:47.235572 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95414144-5e2c-4b7c-bbba-d72ae890b0c5-istio-podinfo\") pod \"e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm\" (UID: \"95414144-5e2c-4b7c-bbba-d72ae890b0c5\") " pod="openshift-ingress/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm" Jul 17 07:56:47.235748 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:47.235588 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95414144-5e2c-4b7c-bbba-d72ae890b0c5-workload-certs\") pod \"e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm\" (UID: \"95414144-5e2c-4b7c-bbba-d72ae890b0c5\") " pod="openshift-ingress/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm" Jul 17 07:56:47.235748 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:47.235607 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95414144-5e2c-4b7c-bbba-d72ae890b0c5-workload-socket\") pod \"e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm\" (UID: \"95414144-5e2c-4b7c-bbba-d72ae890b0c5\") " pod="openshift-ingress/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm" Jul 17 07:56:47.235748 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:47.235658 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95414144-5e2c-4b7c-bbba-d72ae890b0c5-credential-socket\") pod \"e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm\" (UID: \"95414144-5e2c-4b7c-bbba-d72ae890b0c5\") " pod="openshift-ingress/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm" Jul 17 07:56:47.235748 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:47.235690 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95414144-5e2c-4b7c-bbba-d72ae890b0c5-istio-data\") pod \"e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm\" (UID: \"95414144-5e2c-4b7c-bbba-d72ae890b0c5\") " pod="openshift-ingress/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm" Jul 17 07:56:47.235748 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:47.235719 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95414144-5e2c-4b7c-bbba-d72ae890b0c5-istio-token\") pod \"e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm\" (UID: \"95414144-5e2c-4b7c-bbba-d72ae890b0c5\") " pod="openshift-ingress/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm" Jul 17 07:56:47.236092 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:47.236022 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95414144-5e2c-4b7c-bbba-d72ae890b0c5-workload-certs\") pod \"e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm\" (UID: \"95414144-5e2c-4b7c-bbba-d72ae890b0c5\") " pod="openshift-ingress/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm" Jul 17 07:56:47.236092 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:47.236075 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95414144-5e2c-4b7c-bbba-d72ae890b0c5-credential-socket\") pod \"e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm\" (UID: \"95414144-5e2c-4b7c-bbba-d72ae890b0c5\") " pod="openshift-ingress/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm" Jul 17 07:56:47.236216 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:47.236199 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95414144-5e2c-4b7c-bbba-d72ae890b0c5-istio-data\") pod \"e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm\" (UID: \"95414144-5e2c-4b7c-bbba-d72ae890b0c5\") " pod="openshift-ingress/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm" Jul 17 07:56:47.236267 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:47.236198 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95414144-5e2c-4b7c-bbba-d72ae890b0c5-workload-socket\") pod \"e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm\" (UID: \"95414144-5e2c-4b7c-bbba-d72ae890b0c5\") " pod="openshift-ingress/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm" Jul 17 07:56:47.236589 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:47.236551 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95414144-5e2c-4b7c-bbba-d72ae890b0c5-istiod-ca-cert\") pod \"e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm\" (UID: \"95414144-5e2c-4b7c-bbba-d72ae890b0c5\") " pod="openshift-ingress/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm" Jul 17 07:56:47.237988 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:47.237967 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95414144-5e2c-4b7c-bbba-d72ae890b0c5-istio-envoy\") pod \"e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm\" (UID: \"95414144-5e2c-4b7c-bbba-d72ae890b0c5\") " pod="openshift-ingress/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm" Jul 17 07:56:47.238092 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:47.238076 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95414144-5e2c-4b7c-bbba-d72ae890b0c5-istio-podinfo\") pod \"e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm\" (UID: \"95414144-5e2c-4b7c-bbba-d72ae890b0c5\") " pod="openshift-ingress/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm" Jul 17 07:56:47.258148 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:47.258083 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95414144-5e2c-4b7c-bbba-d72ae890b0c5-istio-token\") pod \"e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm\" (UID: \"95414144-5e2c-4b7c-bbba-d72ae890b0c5\") " pod="openshift-ingress/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm" Jul 17 07:56:47.258761 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:47.258740 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7sv9q\" (UniqueName: \"kubernetes.io/projected/95414144-5e2c-4b7c-bbba-d72ae890b0c5-kube-api-access-7sv9q\") pod \"e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm\" (UID: \"95414144-5e2c-4b7c-bbba-d72ae890b0c5\") " pod="openshift-ingress/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm" Jul 17 07:56:47.304454 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:47.304423 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm" Jul 17 07:56:47.454642 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:47.454612 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm"] Jul 17 07:56:47.456480 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:56:47.456441 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95414144_5e2c_4b7c_bbba_d72ae890b0c5.slice/crio-fd2dcc8efdcf2631a2107e5216bf6aa284a81d782aab875aa167bd5c84dbee7b WatchSource:0}: Error finding container fd2dcc8efdcf2631a2107e5216bf6aa284a81d782aab875aa167bd5c84dbee7b: Status 404 returned error can't find the container with id fd2dcc8efdcf2631a2107e5216bf6aa284a81d782aab875aa167bd5c84dbee7b Jul 17 07:56:47.458609 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:47.458575 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:56:47.458744 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:47.458658 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:56:47.458744 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:47.458701 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:56:47.767050 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:47.767011 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm" event={"ID":"95414144-5e2c-4b7c-bbba-d72ae890b0c5","Type":"ContainerStarted","Data":"60dc10142c336435ff653e731c0a575777a07d0e91e7581288fa8c5c1a1fcedd"} Jul 17 07:56:47.767249 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:47.767074 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm" event={"ID":"95414144-5e2c-4b7c-bbba-d72ae890b0c5","Type":"ContainerStarted","Data":"fd2dcc8efdcf2631a2107e5216bf6aa284a81d782aab875aa167bd5c84dbee7b"} Jul 17 07:56:47.791504 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:47.791408 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm" podStartSLOduration=1.791391454 podStartE2EDuration="1.791391454s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 07:56:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:56:47.788935212 +0000 UTC m=+2712.273624188" watchObservedRunningTime="2026-07-17 07:56:47.791391454 +0000 UTC m=+2712.276080427" Jul 17 07:56:48.304862 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:48.304832 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm" Jul 17 07:56:48.310578 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:48.310549 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm" Jul 17 07:56:48.773934 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:48.773905 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm" Jul 17 07:56:48.774783 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:48.774763 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm" Jul 17 07:56:57.609084 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:57.608201 2568 patch_prober.go:28] interesting pod/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": read tcp 10.132.0.2:55106->10.132.0.64:15021: read: connection reset by peer" start-of-body= Jul 17 07:56:57.609084 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:57.608311 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw" podUID="5f1bb1bb-0203-46c6-9089-429977417047" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": read tcp 10.132.0.2:55106->10.132.0.64:15021: read: connection reset by peer" Jul 17 07:56:57.834163 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:57.834077 2568 generic.go:356] "Generic (PLEG): container finished" podID="5f1bb1bb-0203-46c6-9089-429977417047" containerID="c669078c08ba82d0c39af3230464ac33a6e03f705b535d154ce4c25d21e51846" exitCode=0 Jul 17 07:56:57.834337 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:57.834162 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw" event={"ID":"5f1bb1bb-0203-46c6-9089-429977417047","Type":"ContainerDied","Data":"c669078c08ba82d0c39af3230464ac33a6e03f705b535d154ce4c25d21e51846"} Jul 17 07:56:57.834613 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:57.834598 2568 scope.go:117] "RemoveContainer" containerID="c669078c08ba82d0c39af3230464ac33a6e03f705b535d154ce4c25d21e51846" Jul 17 07:56:57.835634 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:57.835597 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:56:57.835739 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:57.835688 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:56:57.835739 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:57.835726 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:56:58.842269 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:58.842237 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw" event={"ID":"5f1bb1bb-0203-46c6-9089-429977417047","Type":"ContainerStarted","Data":"a0f70a8435bb991bb44708208d4f2a22125712c7c9e5cdbd0db7ff8cb01bd784"} Jul 17 07:56:58.991230 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:58.991194 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw" Jul 17 07:56:58.991620 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:58.991592 2568 patch_prober.go:28] interesting pod/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 17 07:56:58.991789 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:58.991763 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw" podUID="5f1bb1bb-0203-46c6-9089-429977417047" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 17 07:56:59.991146 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:59.991100 2568 patch_prober.go:28] interesting pod/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 17 07:56:59.991710 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:56:59.991183 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw" podUID="5f1bb1bb-0203-46c6-9089-429977417047" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 17 07:57:00.992144 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:57:00.991675 2568 patch_prober.go:28] interesting pod/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 17 07:57:00.992144 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:57:00.991742 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw" podUID="5f1bb1bb-0203-46c6-9089-429977417047" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 17 07:57:01.991452 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:57:01.991406 2568 patch_prober.go:28] interesting pod/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 17 07:57:01.991774 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:57:01.991738 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw" podUID="5f1bb1bb-0203-46c6-9089-429977417047" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 17 07:57:02.994437 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:57:02.994405 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw" Jul 17 07:57:02.994862 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:57:02.994762 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw" Jul 17 07:57:02.995496 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:57:02.995477 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw" Jul 17 07:57:04.354849 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:57:04.353149 2568 patch_prober.go:28] interesting pod/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": read tcp 10.132.0.2:50796->10.132.0.65:15021: read: connection reset by peer" start-of-body= Jul 17 07:57:04.354849 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:57:04.353210 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm" podUID="95414144-5e2c-4b7c-bbba-d72ae890b0c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": read tcp 10.132.0.2:50796->10.132.0.65:15021: read: connection reset by peer" Jul 17 07:57:04.881757 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:57:04.881714 2568 generic.go:356] "Generic (PLEG): container finished" podID="95414144-5e2c-4b7c-bbba-d72ae890b0c5" containerID="60dc10142c336435ff653e731c0a575777a07d0e91e7581288fa8c5c1a1fcedd" exitCode=0 Jul 17 07:57:04.881939 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:57:04.881787 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm" event={"ID":"95414144-5e2c-4b7c-bbba-d72ae890b0c5","Type":"ContainerDied","Data":"60dc10142c336435ff653e731c0a575777a07d0e91e7581288fa8c5c1a1fcedd"} Jul 17 07:57:04.882325 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:57:04.882279 2568 scope.go:117] "RemoveContainer" containerID="60dc10142c336435ff653e731c0a575777a07d0e91e7581288fa8c5c1a1fcedd" Jul 17 07:57:04.883244 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:57:04.883218 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:57:04.883322 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:57:04.883304 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:57:04.883366 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:57:04.883348 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:57:05.892557 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:57:05.892524 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm" event={"ID":"95414144-5e2c-4b7c-bbba-d72ae890b0c5","Type":"ContainerStarted","Data":"dea3663b086fff31f69f1a3527d737ae145886d2838d8b4de3d4ec8fa9a006b8"} Jul 17 07:57:06.305203 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:57:06.305117 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm" Jul 17 07:57:06.305425 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:57:06.305402 2568 patch_prober.go:28] interesting pod/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Jul 17 07:57:06.305473 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:57:06.305453 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm" podUID="95414144-5e2c-4b7c-bbba-d72ae890b0c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Jul 17 07:57:07.305374 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:57:07.305336 2568 patch_prober.go:28] interesting pod/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Jul 17 07:57:07.305785 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:57:07.305391 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm" podUID="95414144-5e2c-4b7c-bbba-d72ae890b0c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Jul 17 07:57:08.305195 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:57:08.305156 2568 patch_prober.go:28] interesting pod/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Jul 17 07:57:08.305470 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:57:08.305212 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm" podUID="95414144-5e2c-4b7c-bbba-d72ae890b0c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Jul 17 07:57:09.304804 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:57:09.304761 2568 patch_prober.go:28] interesting pod/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Jul 17 07:57:09.304981 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:57:09.304826 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm" podUID="95414144-5e2c-4b7c-bbba-d72ae890b0c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Jul 17 07:57:10.307924 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:57:10.307897 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm" Jul 17 07:57:10.308322 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:57:10.308261 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm" Jul 17 07:57:10.309035 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:57:10.309018 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm" Jul 17 07:58:20.427390 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:20.427354 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw"] Jul 17 07:58:20.427824 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:20.427634 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw" podUID="5f1bb1bb-0203-46c6-9089-429977417047" containerName="istio-proxy" containerID="cri-o://a0f70a8435bb991bb44708208d4f2a22125712c7c9e5cdbd0db7ff8cb01bd784" gracePeriod=30 Jul 17 07:58:25.685204 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:25.685182 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw" Jul 17 07:58:25.845689 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:25.845592 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f1bb1bb-0203-46c6-9089-429977417047-istio-envoy\") pod \"5f1bb1bb-0203-46c6-9089-429977417047\" (UID: \"5f1bb1bb-0203-46c6-9089-429977417047\") " Jul 17 07:58:25.845689 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:25.845629 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f1bb1bb-0203-46c6-9089-429977417047-istio-token\") pod \"5f1bb1bb-0203-46c6-9089-429977417047\" (UID: \"5f1bb1bb-0203-46c6-9089-429977417047\") " Jul 17 07:58:25.845689 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:25.845654 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f1bb1bb-0203-46c6-9089-429977417047-workload-certs\") pod \"5f1bb1bb-0203-46c6-9089-429977417047\" (UID: \"5f1bb1bb-0203-46c6-9089-429977417047\") " Jul 17 07:58:25.845689 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:25.845677 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f1bb1bb-0203-46c6-9089-429977417047-workload-socket\") pod \"5f1bb1bb-0203-46c6-9089-429977417047\" (UID: \"5f1bb1bb-0203-46c6-9089-429977417047\") " Jul 17 07:58:25.846036 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:25.845745 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f1bb1bb-0203-46c6-9089-429977417047-credential-socket\") pod \"5f1bb1bb-0203-46c6-9089-429977417047\" (UID: \"5f1bb1bb-0203-46c6-9089-429977417047\") " Jul 17 07:58:25.846036 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:25.845781 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f1bb1bb-0203-46c6-9089-429977417047-istio-podinfo\") pod \"5f1bb1bb-0203-46c6-9089-429977417047\" (UID: \"5f1bb1bb-0203-46c6-9089-429977417047\") " Jul 17 07:58:25.846036 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:25.845808 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hplwl\" (UniqueName: \"kubernetes.io/projected/5f1bb1bb-0203-46c6-9089-429977417047-kube-api-access-hplwl\") pod \"5f1bb1bb-0203-46c6-9089-429977417047\" (UID: \"5f1bb1bb-0203-46c6-9089-429977417047\") " Jul 17 07:58:25.846036 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:25.845878 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f1bb1bb-0203-46c6-9089-429977417047-istiod-ca-cert\") pod \"5f1bb1bb-0203-46c6-9089-429977417047\" (UID: \"5f1bb1bb-0203-46c6-9089-429977417047\") " Jul 17 07:58:25.846036 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:25.845919 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f1bb1bb-0203-46c6-9089-429977417047-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5f1bb1bb-0203-46c6-9089-429977417047" (UID: "5f1bb1bb-0203-46c6-9089-429977417047"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:58:25.846036 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:25.845932 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f1bb1bb-0203-46c6-9089-429977417047-istio-data\") pod \"5f1bb1bb-0203-46c6-9089-429977417047\" (UID: \"5f1bb1bb-0203-46c6-9089-429977417047\") " Jul 17 07:58:25.846391 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:25.846311 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f1bb1bb-0203-46c6-9089-429977417047-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5f1bb1bb-0203-46c6-9089-429977417047" (UID: "5f1bb1bb-0203-46c6-9089-429977417047"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:58:25.846391 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:25.846355 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f1bb1bb-0203-46c6-9089-429977417047-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5f1bb1bb-0203-46c6-9089-429977417047" (UID: "5f1bb1bb-0203-46c6-9089-429977417047"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 07:58:25.846505 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:25.846487 2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f1bb1bb-0203-46c6-9089-429977417047-istiod-ca-cert\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:58:25.846558 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:25.846509 2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f1bb1bb-0203-46c6-9089-429977417047-workload-certs\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:58:25.846558 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:25.846516 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f1bb1bb-0203-46c6-9089-429977417047-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5f1bb1bb-0203-46c6-9089-429977417047" (UID: "5f1bb1bb-0203-46c6-9089-429977417047"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:58:25.846558 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:25.846522 2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f1bb1bb-0203-46c6-9089-429977417047-workload-socket\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:58:25.846879 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:25.846857 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f1bb1bb-0203-46c6-9089-429977417047-istio-data" (OuterVolumeSpecName: "istio-data") pod "5f1bb1bb-0203-46c6-9089-429977417047" (UID: "5f1bb1bb-0203-46c6-9089-429977417047"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:58:25.848011 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:25.847980 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f1bb1bb-0203-46c6-9089-429977417047-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5f1bb1bb-0203-46c6-9089-429977417047" (UID: "5f1bb1bb-0203-46c6-9089-429977417047"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:58:25.848011 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:25.847988 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f1bb1bb-0203-46c6-9089-429977417047-kube-api-access-hplwl" (OuterVolumeSpecName: "kube-api-access-hplwl") pod "5f1bb1bb-0203-46c6-9089-429977417047" (UID: "5f1bb1bb-0203-46c6-9089-429977417047"). InnerVolumeSpecName "kube-api-access-hplwl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 07:58:25.848200 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:25.848185 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f1bb1bb-0203-46c6-9089-429977417047-istio-token" (OuterVolumeSpecName: "istio-token") pod "5f1bb1bb-0203-46c6-9089-429977417047" (UID: "5f1bb1bb-0203-46c6-9089-429977417047"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 07:58:25.848345 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:25.848327 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5f1bb1bb-0203-46c6-9089-429977417047-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5f1bb1bb-0203-46c6-9089-429977417047" (UID: "5f1bb1bb-0203-46c6-9089-429977417047"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 17 07:58:25.947940 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:25.947910 2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f1bb1bb-0203-46c6-9089-429977417047-istio-envoy\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:58:25.947940 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:25.947936 2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f1bb1bb-0203-46c6-9089-429977417047-istio-token\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:58:25.947940 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:25.947946 2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f1bb1bb-0203-46c6-9089-429977417047-credential-socket\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:58:25.948151 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:25.947954 2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f1bb1bb-0203-46c6-9089-429977417047-istio-podinfo\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:58:25.948151 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:25.947963 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hplwl\" (UniqueName: \"kubernetes.io/projected/5f1bb1bb-0203-46c6-9089-429977417047-kube-api-access-hplwl\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:58:25.948151 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:25.947972 2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f1bb1bb-0203-46c6-9089-429977417047-istio-data\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:58:26.341448 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:26.341410 2568 generic.go:356] "Generic (PLEG): container finished" podID="5f1bb1bb-0203-46c6-9089-429977417047" containerID="a0f70a8435bb991bb44708208d4f2a22125712c7c9e5cdbd0db7ff8cb01bd784" exitCode=0 Jul 17 07:58:26.341668 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:26.341480 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw" Jul 17 07:58:26.341668 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:26.341498 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw" event={"ID":"5f1bb1bb-0203-46c6-9089-429977417047","Type":"ContainerDied","Data":"a0f70a8435bb991bb44708208d4f2a22125712c7c9e5cdbd0db7ff8cb01bd784"} Jul 17 07:58:26.341668 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:26.341575 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw" event={"ID":"5f1bb1bb-0203-46c6-9089-429977417047","Type":"ContainerDied","Data":"7c7b2c325d1d77cae46838b92097970e49cf8ce9d994eec35680e07ca8d5b1b7"} Jul 17 07:58:26.341668 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:26.341599 2568 scope.go:117] "RemoveContainer" containerID="a0f70a8435bb991bb44708208d4f2a22125712c7c9e5cdbd0db7ff8cb01bd784" Jul 17 07:58:26.350802 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:26.350780 2568 scope.go:117] "RemoveContainer" containerID="c669078c08ba82d0c39af3230464ac33a6e03f705b535d154ce4c25d21e51846" Jul 17 07:58:26.358882 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:26.358867 2568 scope.go:117] "RemoveContainer" containerID="a0f70a8435bb991bb44708208d4f2a22125712c7c9e5cdbd0db7ff8cb01bd784" Jul 17 07:58:26.359117 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:58:26.359100 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0f70a8435bb991bb44708208d4f2a22125712c7c9e5cdbd0db7ff8cb01bd784\": container with ID starting with a0f70a8435bb991bb44708208d4f2a22125712c7c9e5cdbd0db7ff8cb01bd784 not found: ID does not exist" containerID="a0f70a8435bb991bb44708208d4f2a22125712c7c9e5cdbd0db7ff8cb01bd784" Jul 17 07:58:26.359171 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:26.359123 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0f70a8435bb991bb44708208d4f2a22125712c7c9e5cdbd0db7ff8cb01bd784"} err="failed to get container status \"a0f70a8435bb991bb44708208d4f2a22125712c7c9e5cdbd0db7ff8cb01bd784\": rpc error: code = NotFound desc = could not find container \"a0f70a8435bb991bb44708208d4f2a22125712c7c9e5cdbd0db7ff8cb01bd784\": container with ID starting with a0f70a8435bb991bb44708208d4f2a22125712c7c9e5cdbd0db7ff8cb01bd784 not found: ID does not exist" Jul 17 07:58:26.359171 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:26.359138 2568 scope.go:117] "RemoveContainer" containerID="c669078c08ba82d0c39af3230464ac33a6e03f705b535d154ce4c25d21e51846" Jul 17 07:58:26.359380 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:58:26.359364 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c669078c08ba82d0c39af3230464ac33a6e03f705b535d154ce4c25d21e51846\": container with ID starting with c669078c08ba82d0c39af3230464ac33a6e03f705b535d154ce4c25d21e51846 not found: ID does not exist" containerID="c669078c08ba82d0c39af3230464ac33a6e03f705b535d154ce4c25d21e51846" Jul 17 07:58:26.359435 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:26.359385 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c669078c08ba82d0c39af3230464ac33a6e03f705b535d154ce4c25d21e51846"} err="failed to get container status \"c669078c08ba82d0c39af3230464ac33a6e03f705b535d154ce4c25d21e51846\": rpc error: code = NotFound desc = could not find container \"c669078c08ba82d0c39af3230464ac33a6e03f705b535d154ce4c25d21e51846\": container with ID starting with c669078c08ba82d0c39af3230464ac33a6e03f705b535d154ce4c25d21e51846 not found: ID does not exist" Jul 17 07:58:26.365394 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:26.365372 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw"] Jul 17 07:58:26.370383 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:26.370362 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-1f268b-openshift-default-77b64f666c-7r7lw"] Jul 17 07:58:28.000138 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:28.000105 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f1bb1bb-0203-46c6-9089-429977417047" path="/var/lib/kubelet/pods/5f1bb1bb-0203-46c6-9089-429977417047/volumes" Jul 17 07:58:46.681179 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:46.681146 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm"] Jul 17 07:58:46.681611 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:46.681396 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm" podUID="95414144-5e2c-4b7c-bbba-d72ae890b0c5" containerName="istio-proxy" containerID="cri-o://dea3663b086fff31f69f1a3527d737ae145886d2838d8b4de3d4ec8fa9a006b8" gracePeriod=30 Jul 17 07:58:47.623409 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:47.623365 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp"] Jul 17 07:58:47.623777 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:47.623765 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f1bb1bb-0203-46c6-9089-429977417047" containerName="istio-proxy" Jul 17 07:58:47.623818 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:47.623779 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f1bb1bb-0203-46c6-9089-429977417047" containerName="istio-proxy" Jul 17 07:58:47.623868 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:47.623845 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f1bb1bb-0203-46c6-9089-429977417047" containerName="istio-proxy" Jul 17 07:58:47.623868 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:47.623855 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f1bb1bb-0203-46c6-9089-429977417047" containerName="istio-proxy" Jul 17 07:58:47.626940 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:47.626923 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp" Jul 17 07:58:47.629481 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:47.629457 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-5cf58d-openshift-default-dockercfg-pq4mp\"" Jul 17 07:58:47.640074 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:47.640050 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp"] Jul 17 07:58:47.727006 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:47.726970 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d8af748-4c20-4576-87a2-29f50378e09b-istio-podinfo\") pod \"e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp\" (UID: \"1d8af748-4c20-4576-87a2-29f50378e09b\") " pod="openshift-ingress/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp" Jul 17 07:58:47.727006 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:47.727008 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d8af748-4c20-4576-87a2-29f50378e09b-istio-token\") pod \"e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp\" (UID: \"1d8af748-4c20-4576-87a2-29f50378e09b\") " pod="openshift-ingress/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp" Jul 17 07:58:47.727514 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:47.727033 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6d9l\" (UniqueName: \"kubernetes.io/projected/1d8af748-4c20-4576-87a2-29f50378e09b-kube-api-access-s6d9l\") pod \"e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp\" (UID: \"1d8af748-4c20-4576-87a2-29f50378e09b\") " pod="openshift-ingress/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp" Jul 17 07:58:47.727514 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:47.727119 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d8af748-4c20-4576-87a2-29f50378e09b-istio-envoy\") pod \"e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp\" (UID: \"1d8af748-4c20-4576-87a2-29f50378e09b\") " pod="openshift-ingress/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp" Jul 17 07:58:47.727514 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:47.727200 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d8af748-4c20-4576-87a2-29f50378e09b-credential-socket\") pod \"e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp\" (UID: \"1d8af748-4c20-4576-87a2-29f50378e09b\") " pod="openshift-ingress/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp" Jul 17 07:58:47.727514 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:47.727231 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d8af748-4c20-4576-87a2-29f50378e09b-istio-data\") pod \"e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp\" (UID: \"1d8af748-4c20-4576-87a2-29f50378e09b\") " pod="openshift-ingress/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp" Jul 17 07:58:47.727514 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:47.727275 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d8af748-4c20-4576-87a2-29f50378e09b-istiod-ca-cert\") pod \"e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp\" (UID: \"1d8af748-4c20-4576-87a2-29f50378e09b\") " pod="openshift-ingress/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp" Jul 17 07:58:47.727514 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:47.727364 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d8af748-4c20-4576-87a2-29f50378e09b-workload-socket\") pod \"e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp\" (UID: \"1d8af748-4c20-4576-87a2-29f50378e09b\") " pod="openshift-ingress/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp" Jul 17 07:58:47.727514 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:47.727414 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d8af748-4c20-4576-87a2-29f50378e09b-workload-certs\") pod \"e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp\" (UID: \"1d8af748-4c20-4576-87a2-29f50378e09b\") " pod="openshift-ingress/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp" Jul 17 07:58:47.827858 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:47.827828 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d8af748-4c20-4576-87a2-29f50378e09b-workload-certs\") pod \"e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp\" (UID: \"1d8af748-4c20-4576-87a2-29f50378e09b\") " pod="openshift-ingress/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp" Jul 17 07:58:47.828041 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:47.827871 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d8af748-4c20-4576-87a2-29f50378e09b-istio-podinfo\") pod \"e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp\" (UID: \"1d8af748-4c20-4576-87a2-29f50378e09b\") " pod="openshift-ingress/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp" Jul 17 07:58:47.828041 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:47.827889 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d8af748-4c20-4576-87a2-29f50378e09b-istio-token\") pod \"e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp\" (UID: \"1d8af748-4c20-4576-87a2-29f50378e09b\") " pod="openshift-ingress/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp" Jul 17 07:58:47.828041 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:47.827914 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6d9l\" (UniqueName: \"kubernetes.io/projected/1d8af748-4c20-4576-87a2-29f50378e09b-kube-api-access-s6d9l\") pod \"e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp\" (UID: \"1d8af748-4c20-4576-87a2-29f50378e09b\") " pod="openshift-ingress/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp" Jul 17 07:58:47.828041 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:47.827948 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d8af748-4c20-4576-87a2-29f50378e09b-istio-envoy\") pod \"e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp\" (UID: \"1d8af748-4c20-4576-87a2-29f50378e09b\") " pod="openshift-ingress/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp" Jul 17 07:58:47.828041 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:47.827997 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d8af748-4c20-4576-87a2-29f50378e09b-credential-socket\") pod \"e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp\" (UID: \"1d8af748-4c20-4576-87a2-29f50378e09b\") " pod="openshift-ingress/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp" Jul 17 07:58:47.828041 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:47.828022 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d8af748-4c20-4576-87a2-29f50378e09b-istio-data\") pod \"e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp\" (UID: \"1d8af748-4c20-4576-87a2-29f50378e09b\") " pod="openshift-ingress/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp" Jul 17 07:58:47.828417 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:47.828058 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d8af748-4c20-4576-87a2-29f50378e09b-istiod-ca-cert\") pod \"e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp\" (UID: \"1d8af748-4c20-4576-87a2-29f50378e09b\") " pod="openshift-ingress/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp" Jul 17 07:58:47.828417 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:47.828098 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d8af748-4c20-4576-87a2-29f50378e09b-workload-socket\") pod \"e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp\" (UID: \"1d8af748-4c20-4576-87a2-29f50378e09b\") " pod="openshift-ingress/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp" Jul 17 07:58:47.828417 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:47.828323 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d8af748-4c20-4576-87a2-29f50378e09b-workload-certs\") pod \"e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp\" (UID: \"1d8af748-4c20-4576-87a2-29f50378e09b\") " pod="openshift-ingress/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp" Jul 17 07:58:47.828525 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:47.828476 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d8af748-4c20-4576-87a2-29f50378e09b-credential-socket\") pod \"e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp\" (UID: \"1d8af748-4c20-4576-87a2-29f50378e09b\") " pod="openshift-ingress/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp" Jul 17 07:58:47.828623 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:47.828602 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d8af748-4c20-4576-87a2-29f50378e09b-workload-socket\") pod \"e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp\" (UID: \"1d8af748-4c20-4576-87a2-29f50378e09b\") " pod="openshift-ingress/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp" Jul 17 07:58:47.828679 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:47.828627 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d8af748-4c20-4576-87a2-29f50378e09b-istio-data\") pod \"e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp\" (UID: \"1d8af748-4c20-4576-87a2-29f50378e09b\") " pod="openshift-ingress/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp" Jul 17 07:58:47.828821 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:47.828805 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d8af748-4c20-4576-87a2-29f50378e09b-istiod-ca-cert\") pod \"e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp\" (UID: \"1d8af748-4c20-4576-87a2-29f50378e09b\") " pod="openshift-ingress/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp" Jul 17 07:58:47.830307 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:47.830270 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d8af748-4c20-4576-87a2-29f50378e09b-istio-envoy\") pod \"e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp\" (UID: \"1d8af748-4c20-4576-87a2-29f50378e09b\") " pod="openshift-ingress/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp" Jul 17 07:58:47.830489 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:47.830470 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d8af748-4c20-4576-87a2-29f50378e09b-istio-podinfo\") pod \"e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp\" (UID: \"1d8af748-4c20-4576-87a2-29f50378e09b\") " pod="openshift-ingress/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp" Jul 17 07:58:47.836790 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:47.836766 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d8af748-4c20-4576-87a2-29f50378e09b-istio-token\") pod \"e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp\" (UID: \"1d8af748-4c20-4576-87a2-29f50378e09b\") " pod="openshift-ingress/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp" Jul 17 07:58:47.837602 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:47.837565 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6d9l\" (UniqueName: \"kubernetes.io/projected/1d8af748-4c20-4576-87a2-29f50378e09b-kube-api-access-s6d9l\") pod \"e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp\" (UID: \"1d8af748-4c20-4576-87a2-29f50378e09b\") " pod="openshift-ingress/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp" Jul 17 07:58:47.937828 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:47.937728 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp" Jul 17 07:58:48.079669 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:48.079644 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp"] Jul 17 07:58:48.081205 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:58:48.081179 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d8af748_4c20_4576_87a2_29f50378e09b.slice/crio-33fd6bbd8b7649b1889d633f589471f6f285592559056171a2772c89dfd72fd3 WatchSource:0}: Error finding container 33fd6bbd8b7649b1889d633f589471f6f285592559056171a2772c89dfd72fd3: Status 404 returned error can't find the container with id 33fd6bbd8b7649b1889d633f589471f6f285592559056171a2772c89dfd72fd3 Jul 17 07:58:48.083314 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:48.083267 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:58:48.083378 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:48.083350 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:58:48.083416 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:48.083379 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:58:48.462987 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:48.462950 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp" event={"ID":"1d8af748-4c20-4576-87a2-29f50378e09b","Type":"ContainerStarted","Data":"adfb1a541635ef777ccef393f4881291b43da75a0ae41e919f6d68d7433111e0"} Jul 17 07:58:48.463170 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:48.463005 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp" event={"ID":"1d8af748-4c20-4576-87a2-29f50378e09b","Type":"ContainerStarted","Data":"33fd6bbd8b7649b1889d633f589471f6f285592559056171a2772c89dfd72fd3"} Jul 17 07:58:48.487017 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:48.486970 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp" podStartSLOduration=1.486956017 podStartE2EDuration="1.486956017s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 07:58:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:58:48.486364916 +0000 UTC m=+2832.971053894" watchObservedRunningTime="2026-07-17 07:58:48.486956017 +0000 UTC m=+2832.971644990" Jul 17 07:58:48.938588 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:48.938550 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp" Jul 17 07:58:49.943766 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:49.943738 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp" Jul 17 07:58:50.473817 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:50.473783 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp" Jul 17 07:58:50.474795 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:50.474773 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp" Jul 17 07:58:51.947347 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:51.947278 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm" Jul 17 07:58:52.069190 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:52.069158 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95414144-5e2c-4b7c-bbba-d72ae890b0c5-istio-podinfo\") pod \"95414144-5e2c-4b7c-bbba-d72ae890b0c5\" (UID: \"95414144-5e2c-4b7c-bbba-d72ae890b0c5\") " Jul 17 07:58:52.069190 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:52.069192 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95414144-5e2c-4b7c-bbba-d72ae890b0c5-istiod-ca-cert\") pod \"95414144-5e2c-4b7c-bbba-d72ae890b0c5\" (UID: \"95414144-5e2c-4b7c-bbba-d72ae890b0c5\") " Jul 17 07:58:52.069444 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:52.069212 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95414144-5e2c-4b7c-bbba-d72ae890b0c5-istio-envoy\") pod \"95414144-5e2c-4b7c-bbba-d72ae890b0c5\" (UID: \"95414144-5e2c-4b7c-bbba-d72ae890b0c5\") " Jul 17 07:58:52.069444 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:52.069237 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95414144-5e2c-4b7c-bbba-d72ae890b0c5-istio-data\") pod \"95414144-5e2c-4b7c-bbba-d72ae890b0c5\" (UID: \"95414144-5e2c-4b7c-bbba-d72ae890b0c5\") " Jul 17 07:58:52.069444 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:52.069264 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95414144-5e2c-4b7c-bbba-d72ae890b0c5-workload-socket\") pod \"95414144-5e2c-4b7c-bbba-d72ae890b0c5\" (UID: \"95414144-5e2c-4b7c-bbba-d72ae890b0c5\") " Jul 17 07:58:52.069444 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:52.069357 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95414144-5e2c-4b7c-bbba-d72ae890b0c5-workload-certs\") pod \"95414144-5e2c-4b7c-bbba-d72ae890b0c5\" (UID: \"95414144-5e2c-4b7c-bbba-d72ae890b0c5\") " Jul 17 07:58:52.069444 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:52.069403 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95414144-5e2c-4b7c-bbba-d72ae890b0c5-credential-socket\") pod \"95414144-5e2c-4b7c-bbba-d72ae890b0c5\" (UID: \"95414144-5e2c-4b7c-bbba-d72ae890b0c5\") " Jul 17 07:58:52.069444 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:52.069428 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95414144-5e2c-4b7c-bbba-d72ae890b0c5-istio-token\") pod \"95414144-5e2c-4b7c-bbba-d72ae890b0c5\" (UID: \"95414144-5e2c-4b7c-bbba-d72ae890b0c5\") " Jul 17 07:58:52.069761 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:52.069465 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7sv9q\" (UniqueName: \"kubernetes.io/projected/95414144-5e2c-4b7c-bbba-d72ae890b0c5-kube-api-access-7sv9q\") pod \"95414144-5e2c-4b7c-bbba-d72ae890b0c5\" (UID: \"95414144-5e2c-4b7c-bbba-d72ae890b0c5\") " Jul 17 07:58:52.069761 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:52.069600 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95414144-5e2c-4b7c-bbba-d72ae890b0c5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "95414144-5e2c-4b7c-bbba-d72ae890b0c5" (UID: "95414144-5e2c-4b7c-bbba-d72ae890b0c5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:58:52.069761 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:52.069706 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95414144-5e2c-4b7c-bbba-d72ae890b0c5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "95414144-5e2c-4b7c-bbba-d72ae890b0c5" (UID: "95414144-5e2c-4b7c-bbba-d72ae890b0c5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:58:52.069938 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:52.069859 2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95414144-5e2c-4b7c-bbba-d72ae890b0c5-workload-certs\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:58:52.069938 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:52.069881 2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95414144-5e2c-4b7c-bbba-d72ae890b0c5-workload-socket\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:58:52.070170 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:52.070147 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95414144-5e2c-4b7c-bbba-d72ae890b0c5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "95414144-5e2c-4b7c-bbba-d72ae890b0c5" (UID: "95414144-5e2c-4b7c-bbba-d72ae890b0c5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:58:52.070267 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:52.070174 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95414144-5e2c-4b7c-bbba-d72ae890b0c5-istio-data" (OuterVolumeSpecName: "istio-data") pod "95414144-5e2c-4b7c-bbba-d72ae890b0c5" (UID: "95414144-5e2c-4b7c-bbba-d72ae890b0c5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:58:52.070354 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:52.070315 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95414144-5e2c-4b7c-bbba-d72ae890b0c5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "95414144-5e2c-4b7c-bbba-d72ae890b0c5" (UID: "95414144-5e2c-4b7c-bbba-d72ae890b0c5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 07:58:52.071663 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:52.071641 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95414144-5e2c-4b7c-bbba-d72ae890b0c5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "95414144-5e2c-4b7c-bbba-d72ae890b0c5" (UID: "95414144-5e2c-4b7c-bbba-d72ae890b0c5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:58:52.071831 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:52.071814 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95414144-5e2c-4b7c-bbba-d72ae890b0c5-kube-api-access-7sv9q" (OuterVolumeSpecName: "kube-api-access-7sv9q") pod "95414144-5e2c-4b7c-bbba-d72ae890b0c5" (UID: "95414144-5e2c-4b7c-bbba-d72ae890b0c5"). InnerVolumeSpecName "kube-api-access-7sv9q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 07:58:52.071905 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:52.071847 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/95414144-5e2c-4b7c-bbba-d72ae890b0c5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "95414144-5e2c-4b7c-bbba-d72ae890b0c5" (UID: "95414144-5e2c-4b7c-bbba-d72ae890b0c5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 17 07:58:52.071987 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:52.071969 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95414144-5e2c-4b7c-bbba-d72ae890b0c5-istio-token" (OuterVolumeSpecName: "istio-token") pod "95414144-5e2c-4b7c-bbba-d72ae890b0c5" (UID: "95414144-5e2c-4b7c-bbba-d72ae890b0c5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 07:58:52.170691 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:52.170654 2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95414144-5e2c-4b7c-bbba-d72ae890b0c5-istio-podinfo\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:58:52.170691 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:52.170684 2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95414144-5e2c-4b7c-bbba-d72ae890b0c5-istiod-ca-cert\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:58:52.170691 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:52.170693 2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95414144-5e2c-4b7c-bbba-d72ae890b0c5-istio-envoy\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:58:52.170963 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:52.170701 2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95414144-5e2c-4b7c-bbba-d72ae890b0c5-istio-data\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:58:52.170963 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:52.170709 2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95414144-5e2c-4b7c-bbba-d72ae890b0c5-credential-socket\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:58:52.170963 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:52.170718 2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95414144-5e2c-4b7c-bbba-d72ae890b0c5-istio-token\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:58:52.170963 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:52.170727 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7sv9q\" (UniqueName: \"kubernetes.io/projected/95414144-5e2c-4b7c-bbba-d72ae890b0c5-kube-api-access-7sv9q\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:58:52.486134 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:52.486080 2568 generic.go:356] "Generic (PLEG): container finished" podID="95414144-5e2c-4b7c-bbba-d72ae890b0c5" containerID="dea3663b086fff31f69f1a3527d737ae145886d2838d8b4de3d4ec8fa9a006b8" exitCode=0 Jul 17 07:58:52.486134 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:52.486137 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm" Jul 17 07:58:52.486424 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:52.486170 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm" event={"ID":"95414144-5e2c-4b7c-bbba-d72ae890b0c5","Type":"ContainerDied","Data":"dea3663b086fff31f69f1a3527d737ae145886d2838d8b4de3d4ec8fa9a006b8"} Jul 17 07:58:52.486424 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:52.486237 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm" event={"ID":"95414144-5e2c-4b7c-bbba-d72ae890b0c5","Type":"ContainerDied","Data":"fd2dcc8efdcf2631a2107e5216bf6aa284a81d782aab875aa167bd5c84dbee7b"} Jul 17 07:58:52.486424 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:52.486263 2568 scope.go:117] "RemoveContainer" containerID="dea3663b086fff31f69f1a3527d737ae145886d2838d8b4de3d4ec8fa9a006b8" Jul 17 07:58:52.496743 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:52.496724 2568 scope.go:117] "RemoveContainer" containerID="60dc10142c336435ff653e731c0a575777a07d0e91e7581288fa8c5c1a1fcedd" Jul 17 07:58:52.504910 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:52.504895 2568 scope.go:117] "RemoveContainer" containerID="dea3663b086fff31f69f1a3527d737ae145886d2838d8b4de3d4ec8fa9a006b8" Jul 17 07:58:52.505152 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:58:52.505135 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dea3663b086fff31f69f1a3527d737ae145886d2838d8b4de3d4ec8fa9a006b8\": container with ID starting with dea3663b086fff31f69f1a3527d737ae145886d2838d8b4de3d4ec8fa9a006b8 not found: ID does not exist" containerID="dea3663b086fff31f69f1a3527d737ae145886d2838d8b4de3d4ec8fa9a006b8" Jul 17 07:58:52.505196 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:52.505158 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dea3663b086fff31f69f1a3527d737ae145886d2838d8b4de3d4ec8fa9a006b8"} err="failed to get container status \"dea3663b086fff31f69f1a3527d737ae145886d2838d8b4de3d4ec8fa9a006b8\": rpc error: code = NotFound desc = could not find container \"dea3663b086fff31f69f1a3527d737ae145886d2838d8b4de3d4ec8fa9a006b8\": container with ID starting with dea3663b086fff31f69f1a3527d737ae145886d2838d8b4de3d4ec8fa9a006b8 not found: ID does not exist" Jul 17 07:58:52.505196 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:52.505173 2568 scope.go:117] "RemoveContainer" containerID="60dc10142c336435ff653e731c0a575777a07d0e91e7581288fa8c5c1a1fcedd" Jul 17 07:58:52.505437 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:58:52.505419 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60dc10142c336435ff653e731c0a575777a07d0e91e7581288fa8c5c1a1fcedd\": container with ID starting with 60dc10142c336435ff653e731c0a575777a07d0e91e7581288fa8c5c1a1fcedd not found: ID does not exist" containerID="60dc10142c336435ff653e731c0a575777a07d0e91e7581288fa8c5c1a1fcedd" Jul 17 07:58:52.505484 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:52.505443 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60dc10142c336435ff653e731c0a575777a07d0e91e7581288fa8c5c1a1fcedd"} err="failed to get container status \"60dc10142c336435ff653e731c0a575777a07d0e91e7581288fa8c5c1a1fcedd\": rpc error: code = NotFound desc = could not find container \"60dc10142c336435ff653e731c0a575777a07d0e91e7581288fa8c5c1a1fcedd\": container with ID starting with 60dc10142c336435ff653e731c0a575777a07d0e91e7581288fa8c5c1a1fcedd not found: ID does not exist" Jul 17 07:58:52.514823 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:52.514800 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm"] Jul 17 07:58:52.517022 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:52.517004 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-cab91c-openshift-default-7b4b59c7b5-5mkvm"] Jul 17 07:58:54.000280 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:54.000245 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95414144-5e2c-4b7c-bbba-d72ae890b0c5" path="/var/lib/kubelet/pods/95414144-5e2c-4b7c-bbba-d72ae890b0c5/volumes" Jul 17 07:58:55.891234 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:55.891204 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv"] Jul 17 07:58:55.891637 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:55.891623 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95414144-5e2c-4b7c-bbba-d72ae890b0c5" containerName="istio-proxy" Jul 17 07:58:55.891691 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:55.891639 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="95414144-5e2c-4b7c-bbba-d72ae890b0c5" containerName="istio-proxy" Jul 17 07:58:55.891691 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:55.891650 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f1bb1bb-0203-46c6-9089-429977417047" containerName="istio-proxy" Jul 17 07:58:55.891691 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:55.891656 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f1bb1bb-0203-46c6-9089-429977417047" containerName="istio-proxy" Jul 17 07:58:55.891790 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:55.891720 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="95414144-5e2c-4b7c-bbba-d72ae890b0c5" containerName="istio-proxy" Jul 17 07:58:55.891790 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:55.891730 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="95414144-5e2c-4b7c-bbba-d72ae890b0c5" containerName="istio-proxy" Jul 17 07:58:55.897816 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:55.897772 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv" Jul 17 07:58:55.901328 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:55.901274 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-38e02d-openshift-default-dockercfg-qgpk7\"" Jul 17 07:58:55.910556 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:55.910530 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv"] Jul 17 07:58:56.007146 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:56.007113 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8dvj\" (UniqueName: \"kubernetes.io/projected/5b7ae261-290d-48a8-9e0b-635e9abe3a4c-kube-api-access-g8dvj\") pod \"e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv\" (UID: \"5b7ae261-290d-48a8-9e0b-635e9abe3a4c\") " pod="openshift-ingress/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv" Jul 17 07:58:56.007350 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:56.007184 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5b7ae261-290d-48a8-9e0b-635e9abe3a4c-istio-podinfo\") pod \"e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv\" (UID: \"5b7ae261-290d-48a8-9e0b-635e9abe3a4c\") " pod="openshift-ingress/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv" Jul 17 07:58:56.007350 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:56.007211 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b7ae261-290d-48a8-9e0b-635e9abe3a4c-istio-token\") pod \"e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv\" (UID: \"5b7ae261-290d-48a8-9e0b-635e9abe3a4c\") " pod="openshift-ingress/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv" Jul 17 07:58:56.007350 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:56.007274 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5b7ae261-290d-48a8-9e0b-635e9abe3a4c-istio-envoy\") pod \"e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv\" (UID: \"5b7ae261-290d-48a8-9e0b-635e9abe3a4c\") " pod="openshift-ingress/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv" Jul 17 07:58:56.007665 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:56.007388 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b7ae261-290d-48a8-9e0b-635e9abe3a4c-credential-socket\") pod \"e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv\" (UID: \"5b7ae261-290d-48a8-9e0b-635e9abe3a4c\") " pod="openshift-ingress/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv" Jul 17 07:58:56.007665 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:56.007426 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b7ae261-290d-48a8-9e0b-635e9abe3a4c-workload-certs\") pod \"e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv\" (UID: \"5b7ae261-290d-48a8-9e0b-635e9abe3a4c\") " pod="openshift-ingress/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv" Jul 17 07:58:56.007665 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:56.007497 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5b7ae261-290d-48a8-9e0b-635e9abe3a4c-istiod-ca-cert\") pod \"e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv\" (UID: \"5b7ae261-290d-48a8-9e0b-635e9abe3a4c\") " pod="openshift-ingress/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv" Jul 17 07:58:56.007665 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:56.007540 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b7ae261-290d-48a8-9e0b-635e9abe3a4c-workload-socket\") pod \"e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv\" (UID: \"5b7ae261-290d-48a8-9e0b-635e9abe3a4c\") " pod="openshift-ingress/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv" Jul 17 07:58:56.007665 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:56.007571 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5b7ae261-290d-48a8-9e0b-635e9abe3a4c-istio-data\") pod \"e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv\" (UID: \"5b7ae261-290d-48a8-9e0b-635e9abe3a4c\") " pod="openshift-ingress/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv" Jul 17 07:58:56.108943 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:56.108895 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5b7ae261-290d-48a8-9e0b-635e9abe3a4c-istio-podinfo\") pod \"e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv\" (UID: \"5b7ae261-290d-48a8-9e0b-635e9abe3a4c\") " pod="openshift-ingress/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv" Jul 17 07:58:56.108943 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:56.108946 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b7ae261-290d-48a8-9e0b-635e9abe3a4c-istio-token\") pod \"e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv\" (UID: \"5b7ae261-290d-48a8-9e0b-635e9abe3a4c\") " pod="openshift-ingress/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv" Jul 17 07:58:56.109143 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:56.109054 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5b7ae261-290d-48a8-9e0b-635e9abe3a4c-istio-envoy\") pod \"e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv\" (UID: \"5b7ae261-290d-48a8-9e0b-635e9abe3a4c\") " pod="openshift-ingress/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv" Jul 17 07:58:56.109143 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:56.109112 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b7ae261-290d-48a8-9e0b-635e9abe3a4c-credential-socket\") pod \"e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv\" (UID: \"5b7ae261-290d-48a8-9e0b-635e9abe3a4c\") " pod="openshift-ingress/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv" Jul 17 07:58:56.109221 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:56.109154 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b7ae261-290d-48a8-9e0b-635e9abe3a4c-workload-certs\") pod \"e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv\" (UID: \"5b7ae261-290d-48a8-9e0b-635e9abe3a4c\") " pod="openshift-ingress/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv" Jul 17 07:58:56.109221 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:56.109212 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5b7ae261-290d-48a8-9e0b-635e9abe3a4c-istiod-ca-cert\") pod \"e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv\" (UID: \"5b7ae261-290d-48a8-9e0b-635e9abe3a4c\") " pod="openshift-ingress/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv" Jul 17 07:58:56.109372 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:56.109279 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b7ae261-290d-48a8-9e0b-635e9abe3a4c-workload-socket\") pod \"e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv\" (UID: \"5b7ae261-290d-48a8-9e0b-635e9abe3a4c\") " pod="openshift-ingress/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv" Jul 17 07:58:56.109372 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:56.109355 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5b7ae261-290d-48a8-9e0b-635e9abe3a4c-istio-data\") pod \"e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv\" (UID: \"5b7ae261-290d-48a8-9e0b-635e9abe3a4c\") " pod="openshift-ingress/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv" Jul 17 07:58:56.109513 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:56.109486 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8dvj\" (UniqueName: \"kubernetes.io/projected/5b7ae261-290d-48a8-9e0b-635e9abe3a4c-kube-api-access-g8dvj\") pod \"e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv\" (UID: \"5b7ae261-290d-48a8-9e0b-635e9abe3a4c\") " pod="openshift-ingress/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv" Jul 17 07:58:56.109575 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:56.109552 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b7ae261-290d-48a8-9e0b-635e9abe3a4c-credential-socket\") pod \"e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv\" (UID: \"5b7ae261-290d-48a8-9e0b-635e9abe3a4c\") " pod="openshift-ingress/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv" Jul 17 07:58:56.109629 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:56.109581 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b7ae261-290d-48a8-9e0b-635e9abe3a4c-workload-certs\") pod \"e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv\" (UID: \"5b7ae261-290d-48a8-9e0b-635e9abe3a4c\") " pod="openshift-ingress/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv" Jul 17 07:58:56.109692 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:56.109663 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5b7ae261-290d-48a8-9e0b-635e9abe3a4c-istio-data\") pod \"e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv\" (UID: \"5b7ae261-290d-48a8-9e0b-635e9abe3a4c\") " pod="openshift-ingress/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv" Jul 17 07:58:56.109741 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:56.109699 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b7ae261-290d-48a8-9e0b-635e9abe3a4c-workload-socket\") pod \"e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv\" (UID: \"5b7ae261-290d-48a8-9e0b-635e9abe3a4c\") " pod="openshift-ingress/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv" Jul 17 07:58:56.109904 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:56.109885 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5b7ae261-290d-48a8-9e0b-635e9abe3a4c-istiod-ca-cert\") pod \"e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv\" (UID: \"5b7ae261-290d-48a8-9e0b-635e9abe3a4c\") " pod="openshift-ingress/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv" Jul 17 07:58:56.111408 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:56.111388 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5b7ae261-290d-48a8-9e0b-635e9abe3a4c-istio-envoy\") pod \"e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv\" (UID: \"5b7ae261-290d-48a8-9e0b-635e9abe3a4c\") " pod="openshift-ingress/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv" Jul 17 07:58:56.111504 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:56.111391 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5b7ae261-290d-48a8-9e0b-635e9abe3a4c-istio-podinfo\") pod \"e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv\" (UID: \"5b7ae261-290d-48a8-9e0b-635e9abe3a4c\") " pod="openshift-ingress/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv" Jul 17 07:58:56.118399 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:56.118378 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b7ae261-290d-48a8-9e0b-635e9abe3a4c-istio-token\") pod \"e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv\" (UID: \"5b7ae261-290d-48a8-9e0b-635e9abe3a4c\") " pod="openshift-ingress/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv" Jul 17 07:58:56.121615 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:56.121594 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8dvj\" (UniqueName: \"kubernetes.io/projected/5b7ae261-290d-48a8-9e0b-635e9abe3a4c-kube-api-access-g8dvj\") pod \"e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv\" (UID: \"5b7ae261-290d-48a8-9e0b-635e9abe3a4c\") " pod="openshift-ingress/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv" Jul 17 07:58:56.210332 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:56.210224 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv" Jul 17 07:58:56.363826 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:56.363626 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv"] Jul 17 07:58:56.366912 ip-10-0-138-233 kubenswrapper[2568]: W0717 07:58:56.366885 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b7ae261_290d_48a8_9e0b_635e9abe3a4c.slice/crio-716df93ed843f23e31a5aaa6b54fda232029558940175038d0b671bfdb47192f WatchSource:0}: Error finding container 716df93ed843f23e31a5aaa6b54fda232029558940175038d0b671bfdb47192f: Status 404 returned error can't find the container with id 716df93ed843f23e31a5aaa6b54fda232029558940175038d0b671bfdb47192f Jul 17 07:58:56.369706 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:56.369667 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:58:56.369841 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:56.369742 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:58:56.369841 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:56.369768 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:58:56.512319 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:56.512210 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv" event={"ID":"5b7ae261-290d-48a8-9e0b-635e9abe3a4c","Type":"ContainerStarted","Data":"8a23e70a610c973fe9025404d2148f0b268ee3b68849e56bb9c44cd7605740e8"} Jul 17 07:58:56.512319 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:56.512279 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv" event={"ID":"5b7ae261-290d-48a8-9e0b-635e9abe3a4c","Type":"ContainerStarted","Data":"716df93ed843f23e31a5aaa6b54fda232029558940175038d0b671bfdb47192f"} Jul 17 07:58:56.535625 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:56.535576 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv" podStartSLOduration=1.535559959 podStartE2EDuration="1.535559959s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 07:58:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 07:58:56.533657736 +0000 UTC m=+2841.018346711" watchObservedRunningTime="2026-07-17 07:58:56.535559959 +0000 UTC m=+2841.020248932" Jul 17 07:58:57.210368 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:57.210327 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv" Jul 17 07:58:57.215557 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:57.215529 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv" Jul 17 07:58:57.517891 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:57.517815 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv" Jul 17 07:58:57.518701 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:58:57.518677 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv" Jul 17 07:59:03.555356 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:03.555324 2568 generic.go:356] "Generic (PLEG): container finished" podID="1d8af748-4c20-4576-87a2-29f50378e09b" containerID="adfb1a541635ef777ccef393f4881291b43da75a0ae41e919f6d68d7433111e0" exitCode=0 Jul 17 07:59:03.555713 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:03.555382 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp" event={"ID":"1d8af748-4c20-4576-87a2-29f50378e09b","Type":"ContainerDied","Data":"adfb1a541635ef777ccef393f4881291b43da75a0ae41e919f6d68d7433111e0"} Jul 17 07:59:03.555782 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:03.555761 2568 scope.go:117] "RemoveContainer" containerID="adfb1a541635ef777ccef393f4881291b43da75a0ae41e919f6d68d7433111e0" Jul 17 07:59:03.556849 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:03.556823 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:59:03.556906 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:03.556889 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:59:03.556941 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:03.556930 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:59:04.565980 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:04.565092 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp" event={"ID":"1d8af748-4c20-4576-87a2-29f50378e09b","Type":"ContainerStarted","Data":"639f70e1f0dfae6b5f5c4b42debb1fb9c0ffc7ffa017dd73b6c1139d152b1c09"} Jul 17 07:59:04.938656 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:04.938613 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp" Jul 17 07:59:04.939807 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:04.939786 2568 patch_prober.go:28] interesting pod/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Jul 17 07:59:04.939891 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:04.939843 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp" podUID="1d8af748-4c20-4576-87a2-29f50378e09b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Jul 17 07:59:05.474340 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:05.474275 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp" Jul 17 07:59:05.939053 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:05.939011 2568 patch_prober.go:28] interesting pod/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Jul 17 07:59:05.939456 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:05.939086 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp" podUID="1d8af748-4c20-4576-87a2-29f50378e09b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Jul 17 07:59:06.938026 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:06.937992 2568 patch_prober.go:28] interesting pod/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Jul 17 07:59:06.938220 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:06.938045 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp" podUID="1d8af748-4c20-4576-87a2-29f50378e09b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Jul 17 07:59:07.938705 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:07.938667 2568 patch_prober.go:28] interesting pod/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Jul 17 07:59:07.939087 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:07.938734 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp" podUID="1d8af748-4c20-4576-87a2-29f50378e09b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Jul 17 07:59:08.942558 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:08.942530 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp" Jul 17 07:59:08.943513 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:08.943494 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp" Jul 17 07:59:13.559512 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:13.559463 2568 patch_prober.go:28] interesting pod/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 17 07:59:13.560004 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:13.559558 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv" podUID="5b7ae261-290d-48a8-9e0b-635e9abe3a4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 17 07:59:15.640871 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:15.640826 2568 generic.go:356] "Generic (PLEG): container finished" podID="5b7ae261-290d-48a8-9e0b-635e9abe3a4c" containerID="8a23e70a610c973fe9025404d2148f0b268ee3b68849e56bb9c44cd7605740e8" exitCode=0 Jul 17 07:59:15.641271 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:15.640899 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv" event={"ID":"5b7ae261-290d-48a8-9e0b-635e9abe3a4c","Type":"ContainerDied","Data":"8a23e70a610c973fe9025404d2148f0b268ee3b68849e56bb9c44cd7605740e8"} Jul 17 07:59:15.641334 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:15.641326 2568 scope.go:117] "RemoveContainer" containerID="8a23e70a610c973fe9025404d2148f0b268ee3b68849e56bb9c44cd7605740e8" Jul 17 07:59:15.642366 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:15.642341 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:59:15.642477 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:15.642408 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:59:15.642477 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:15.642440 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 07:59:16.649805 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:16.649771 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv" event={"ID":"5b7ae261-290d-48a8-9e0b-635e9abe3a4c","Type":"ContainerStarted","Data":"d6e74957bccf92a0cd870fbabc7b8a98bd9053a6637786dd49eb340445d8e0e0"} Jul 17 07:59:17.210709 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:17.210673 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv" Jul 17 07:59:17.210910 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:17.210889 2568 patch_prober.go:28] interesting pod/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Jul 17 07:59:17.210958 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:17.210941 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv" podUID="5b7ae261-290d-48a8-9e0b-635e9abe3a4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Jul 17 07:59:18.210611 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:18.210576 2568 patch_prober.go:28] interesting pod/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Jul 17 07:59:18.211079 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:18.210648 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv" podUID="5b7ae261-290d-48a8-9e0b-635e9abe3a4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Jul 17 07:59:19.211508 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:19.211471 2568 patch_prober.go:28] interesting pod/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Jul 17 07:59:19.211897 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:19.211544 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv" podUID="5b7ae261-290d-48a8-9e0b-635e9abe3a4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Jul 17 07:59:20.211373 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:20.211341 2568 patch_prober.go:28] interesting pod/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Jul 17 07:59:20.211568 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:20.211412 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv" podUID="5b7ae261-290d-48a8-9e0b-635e9abe3a4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Jul 17 07:59:21.213898 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:21.213868 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv" Jul 17 07:59:21.214364 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:21.214141 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv" Jul 17 07:59:21.214918 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:21.214898 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv" Jul 17 07:59:37.336046 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:37.336011 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp"] Jul 17 07:59:37.336578 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:37.336226 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp" podUID="1d8af748-4c20-4576-87a2-29f50378e09b" containerName="istio-proxy" containerID="cri-o://639f70e1f0dfae6b5f5c4b42debb1fb9c0ffc7ffa017dd73b6c1139d152b1c09" gracePeriod=30 Jul 17 07:59:38.944958 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:38.944922 2568 patch_prober.go:28] interesting pod/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 17 07:59:38.945372 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:38.944992 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp" podUID="1d8af748-4c20-4576-87a2-29f50378e09b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 17 07:59:42.594485 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:42.594425 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp" Jul 17 07:59:42.652566 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:42.652536 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d8af748-4c20-4576-87a2-29f50378e09b-istio-token\") pod \"1d8af748-4c20-4576-87a2-29f50378e09b\" (UID: \"1d8af748-4c20-4576-87a2-29f50378e09b\") " Jul 17 07:59:42.652731 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:42.652576 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d8af748-4c20-4576-87a2-29f50378e09b-istio-data\") pod \"1d8af748-4c20-4576-87a2-29f50378e09b\" (UID: \"1d8af748-4c20-4576-87a2-29f50378e09b\") " Jul 17 07:59:42.652731 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:42.652598 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s6d9l\" (UniqueName: \"kubernetes.io/projected/1d8af748-4c20-4576-87a2-29f50378e09b-kube-api-access-s6d9l\") pod \"1d8af748-4c20-4576-87a2-29f50378e09b\" (UID: \"1d8af748-4c20-4576-87a2-29f50378e09b\") " Jul 17 07:59:42.652731 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:42.652628 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d8af748-4c20-4576-87a2-29f50378e09b-istiod-ca-cert\") pod \"1d8af748-4c20-4576-87a2-29f50378e09b\" (UID: \"1d8af748-4c20-4576-87a2-29f50378e09b\") " Jul 17 07:59:42.652731 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:42.652653 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d8af748-4c20-4576-87a2-29f50378e09b-istio-podinfo\") pod \"1d8af748-4c20-4576-87a2-29f50378e09b\" (UID: \"1d8af748-4c20-4576-87a2-29f50378e09b\") " Jul 17 07:59:42.652731 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:42.652720 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d8af748-4c20-4576-87a2-29f50378e09b-credential-socket\") pod \"1d8af748-4c20-4576-87a2-29f50378e09b\" (UID: \"1d8af748-4c20-4576-87a2-29f50378e09b\") " Jul 17 07:59:42.653019 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:42.652801 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d8af748-4c20-4576-87a2-29f50378e09b-istio-envoy\") pod \"1d8af748-4c20-4576-87a2-29f50378e09b\" (UID: \"1d8af748-4c20-4576-87a2-29f50378e09b\") " Jul 17 07:59:42.653019 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:42.652826 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d8af748-4c20-4576-87a2-29f50378e09b-workload-certs\") pod \"1d8af748-4c20-4576-87a2-29f50378e09b\" (UID: \"1d8af748-4c20-4576-87a2-29f50378e09b\") " Jul 17 07:59:42.653019 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:42.652868 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d8af748-4c20-4576-87a2-29f50378e09b-workload-socket\") pod \"1d8af748-4c20-4576-87a2-29f50378e09b\" (UID: \"1d8af748-4c20-4576-87a2-29f50378e09b\") " Jul 17 07:59:42.653170 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:42.653058 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d8af748-4c20-4576-87a2-29f50378e09b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1d8af748-4c20-4576-87a2-29f50378e09b" (UID: "1d8af748-4c20-4576-87a2-29f50378e09b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 07:59:42.653329 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:42.653297 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d8af748-4c20-4576-87a2-29f50378e09b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1d8af748-4c20-4576-87a2-29f50378e09b" (UID: "1d8af748-4c20-4576-87a2-29f50378e09b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:59:42.653480 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:42.653357 2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d8af748-4c20-4576-87a2-29f50378e09b-istiod-ca-cert\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:59:42.653559 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:42.653474 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d8af748-4c20-4576-87a2-29f50378e09b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1d8af748-4c20-4576-87a2-29f50378e09b" (UID: "1d8af748-4c20-4576-87a2-29f50378e09b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:59:42.653559 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:42.653485 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d8af748-4c20-4576-87a2-29f50378e09b-istio-data" (OuterVolumeSpecName: "istio-data") pod "1d8af748-4c20-4576-87a2-29f50378e09b" (UID: "1d8af748-4c20-4576-87a2-29f50378e09b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:59:42.653559 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:42.653500 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d8af748-4c20-4576-87a2-29f50378e09b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1d8af748-4c20-4576-87a2-29f50378e09b" (UID: "1d8af748-4c20-4576-87a2-29f50378e09b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:59:42.654951 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:42.654925 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1d8af748-4c20-4576-87a2-29f50378e09b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1d8af748-4c20-4576-87a2-29f50378e09b" (UID: "1d8af748-4c20-4576-87a2-29f50378e09b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 17 07:59:42.654951 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:42.654939 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d8af748-4c20-4576-87a2-29f50378e09b-istio-token" (OuterVolumeSpecName: "istio-token") pod "1d8af748-4c20-4576-87a2-29f50378e09b" (UID: "1d8af748-4c20-4576-87a2-29f50378e09b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 07:59:42.655082 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:42.654990 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d8af748-4c20-4576-87a2-29f50378e09b-kube-api-access-s6d9l" (OuterVolumeSpecName: "kube-api-access-s6d9l") pod "1d8af748-4c20-4576-87a2-29f50378e09b" (UID: "1d8af748-4c20-4576-87a2-29f50378e09b"). InnerVolumeSpecName "kube-api-access-s6d9l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 07:59:42.655082 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:42.655065 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d8af748-4c20-4576-87a2-29f50378e09b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1d8af748-4c20-4576-87a2-29f50378e09b" (UID: "1d8af748-4c20-4576-87a2-29f50378e09b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 07:59:42.754034 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:42.753995 2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d8af748-4c20-4576-87a2-29f50378e09b-credential-socket\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:59:42.754034 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:42.754029 2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d8af748-4c20-4576-87a2-29f50378e09b-istio-envoy\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:59:42.754034 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:42.754038 2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d8af748-4c20-4576-87a2-29f50378e09b-workload-certs\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:59:42.754271 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:42.754046 2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d8af748-4c20-4576-87a2-29f50378e09b-workload-socket\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:59:42.754271 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:42.754055 2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d8af748-4c20-4576-87a2-29f50378e09b-istio-token\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:59:42.754271 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:42.754063 2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d8af748-4c20-4576-87a2-29f50378e09b-istio-data\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:59:42.754271 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:42.754070 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s6d9l\" (UniqueName: \"kubernetes.io/projected/1d8af748-4c20-4576-87a2-29f50378e09b-kube-api-access-s6d9l\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:59:42.754271 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:42.754079 2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d8af748-4c20-4576-87a2-29f50378e09b-istio-podinfo\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 07:59:42.792453 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:42.792416 2568 generic.go:356] "Generic (PLEG): container finished" podID="1d8af748-4c20-4576-87a2-29f50378e09b" containerID="639f70e1f0dfae6b5f5c4b42debb1fb9c0ffc7ffa017dd73b6c1139d152b1c09" exitCode=0 Jul 17 07:59:42.792602 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:42.792463 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp" event={"ID":"1d8af748-4c20-4576-87a2-29f50378e09b","Type":"ContainerDied","Data":"639f70e1f0dfae6b5f5c4b42debb1fb9c0ffc7ffa017dd73b6c1139d152b1c09"} Jul 17 07:59:42.792602 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:42.792478 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp" Jul 17 07:59:42.792602 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:42.792514 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp" event={"ID":"1d8af748-4c20-4576-87a2-29f50378e09b","Type":"ContainerDied","Data":"33fd6bbd8b7649b1889d633f589471f6f285592559056171a2772c89dfd72fd3"} Jul 17 07:59:42.792602 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:42.792540 2568 scope.go:117] "RemoveContainer" containerID="639f70e1f0dfae6b5f5c4b42debb1fb9c0ffc7ffa017dd73b6c1139d152b1c09" Jul 17 07:59:42.802782 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:42.802766 2568 scope.go:117] "RemoveContainer" containerID="adfb1a541635ef777ccef393f4881291b43da75a0ae41e919f6d68d7433111e0" Jul 17 07:59:42.812736 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:42.812622 2568 scope.go:117] "RemoveContainer" containerID="639f70e1f0dfae6b5f5c4b42debb1fb9c0ffc7ffa017dd73b6c1139d152b1c09" Jul 17 07:59:42.813848 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:59:42.813095 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"639f70e1f0dfae6b5f5c4b42debb1fb9c0ffc7ffa017dd73b6c1139d152b1c09\": container with ID starting with 639f70e1f0dfae6b5f5c4b42debb1fb9c0ffc7ffa017dd73b6c1139d152b1c09 not found: ID does not exist" containerID="639f70e1f0dfae6b5f5c4b42debb1fb9c0ffc7ffa017dd73b6c1139d152b1c09" Jul 17 07:59:42.813848 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:42.813130 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"639f70e1f0dfae6b5f5c4b42debb1fb9c0ffc7ffa017dd73b6c1139d152b1c09"} err="failed to get container status \"639f70e1f0dfae6b5f5c4b42debb1fb9c0ffc7ffa017dd73b6c1139d152b1c09\": rpc error: code = NotFound desc = could not find container \"639f70e1f0dfae6b5f5c4b42debb1fb9c0ffc7ffa017dd73b6c1139d152b1c09\": container with ID starting with 639f70e1f0dfae6b5f5c4b42debb1fb9c0ffc7ffa017dd73b6c1139d152b1c09 not found: ID does not exist" Jul 17 07:59:42.813848 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:42.813169 2568 scope.go:117] "RemoveContainer" containerID="adfb1a541635ef777ccef393f4881291b43da75a0ae41e919f6d68d7433111e0" Jul 17 07:59:42.814081 ip-10-0-138-233 kubenswrapper[2568]: E0717 07:59:42.813928 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"adfb1a541635ef777ccef393f4881291b43da75a0ae41e919f6d68d7433111e0\": container with ID starting with adfb1a541635ef777ccef393f4881291b43da75a0ae41e919f6d68d7433111e0 not found: ID does not exist" containerID="adfb1a541635ef777ccef393f4881291b43da75a0ae41e919f6d68d7433111e0" Jul 17 07:59:42.814081 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:42.813962 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"adfb1a541635ef777ccef393f4881291b43da75a0ae41e919f6d68d7433111e0"} err="failed to get container status \"adfb1a541635ef777ccef393f4881291b43da75a0ae41e919f6d68d7433111e0\": rpc error: code = NotFound desc = could not find container \"adfb1a541635ef777ccef393f4881291b43da75a0ae41e919f6d68d7433111e0\": container with ID starting with adfb1a541635ef777ccef393f4881291b43da75a0ae41e919f6d68d7433111e0 not found: ID does not exist" Jul 17 07:59:42.816551 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:42.816529 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp"] Jul 17 07:59:42.827822 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:42.827796 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-5cf58d-openshift-default-7cb58cf67c-gnzqp"] Jul 17 07:59:44.000301 ip-10-0-138-233 kubenswrapper[2568]: I0717 07:59:44.000260 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d8af748-4c20-4576-87a2-29f50378e09b" path="/var/lib/kubelet/pods/1d8af748-4c20-4576-87a2-29f50378e09b/volumes" Jul 17 08:00:03.584825 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:03.584791 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv"] Jul 17 08:00:03.585252 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:03.585043 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv" podUID="5b7ae261-290d-48a8-9e0b-635e9abe3a4c" containerName="istio-proxy" containerID="cri-o://d6e74957bccf92a0cd870fbabc7b8a98bd9053a6637786dd49eb340445d8e0e0" gracePeriod=30 Jul 17 08:00:04.543144 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:04.543108 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz"] Jul 17 08:00:04.543538 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:04.543524 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d8af748-4c20-4576-87a2-29f50378e09b" containerName="istio-proxy" Jul 17 08:00:04.543585 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:04.543539 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d8af748-4c20-4576-87a2-29f50378e09b" containerName="istio-proxy" Jul 17 08:00:04.543585 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:04.543549 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95414144-5e2c-4b7c-bbba-d72ae890b0c5" containerName="istio-proxy" Jul 17 08:00:04.543585 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:04.543554 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="95414144-5e2c-4b7c-bbba-d72ae890b0c5" containerName="istio-proxy" Jul 17 08:00:04.543585 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:04.543564 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d8af748-4c20-4576-87a2-29f50378e09b" containerName="istio-proxy" Jul 17 08:00:04.543585 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:04.543571 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d8af748-4c20-4576-87a2-29f50378e09b" containerName="istio-proxy" Jul 17 08:00:04.543784 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:04.543629 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d8af748-4c20-4576-87a2-29f50378e09b" containerName="istio-proxy" Jul 17 08:00:04.546963 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:04.546942 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz" Jul 17 08:00:04.551952 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:04.551931 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-d805bd-openshift-default-dockercfg-trc9z\"" Jul 17 08:00:04.563294 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:04.563258 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz"] Jul 17 08:00:04.643437 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:04.643405 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nhmn\" (UniqueName: \"kubernetes.io/projected/d9a11756-b5ff-4f48-bc78-ed0d34e498f1-kube-api-access-7nhmn\") pod \"e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz\" (UID: \"d9a11756-b5ff-4f48-bc78-ed0d34e498f1\") " pod="openshift-ingress/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz" Jul 17 08:00:04.643897 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:04.643461 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9a11756-b5ff-4f48-bc78-ed0d34e498f1-istio-envoy\") pod \"e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz\" (UID: \"d9a11756-b5ff-4f48-bc78-ed0d34e498f1\") " pod="openshift-ingress/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz" Jul 17 08:00:04.643897 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:04.643559 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9a11756-b5ff-4f48-bc78-ed0d34e498f1-workload-certs\") pod \"e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz\" (UID: \"d9a11756-b5ff-4f48-bc78-ed0d34e498f1\") " pod="openshift-ingress/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz" Jul 17 08:00:04.643897 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:04.643612 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9a11756-b5ff-4f48-bc78-ed0d34e498f1-istiod-ca-cert\") pod \"e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz\" (UID: \"d9a11756-b5ff-4f48-bc78-ed0d34e498f1\") " pod="openshift-ingress/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz" Jul 17 08:00:04.643897 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:04.643670 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9a11756-b5ff-4f48-bc78-ed0d34e498f1-credential-socket\") pod \"e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz\" (UID: \"d9a11756-b5ff-4f48-bc78-ed0d34e498f1\") " pod="openshift-ingress/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz" Jul 17 08:00:04.643897 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:04.643695 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9a11756-b5ff-4f48-bc78-ed0d34e498f1-istio-podinfo\") pod \"e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz\" (UID: \"d9a11756-b5ff-4f48-bc78-ed0d34e498f1\") " pod="openshift-ingress/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz" Jul 17 08:00:04.643897 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:04.643749 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9a11756-b5ff-4f48-bc78-ed0d34e498f1-workload-socket\") pod \"e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz\" (UID: \"d9a11756-b5ff-4f48-bc78-ed0d34e498f1\") " pod="openshift-ingress/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz" Jul 17 08:00:04.643897 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:04.643863 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9a11756-b5ff-4f48-bc78-ed0d34e498f1-istio-data\") pod \"e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz\" (UID: \"d9a11756-b5ff-4f48-bc78-ed0d34e498f1\") " pod="openshift-ingress/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz" Jul 17 08:00:04.644156 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:04.643906 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9a11756-b5ff-4f48-bc78-ed0d34e498f1-istio-token\") pod \"e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz\" (UID: \"d9a11756-b5ff-4f48-bc78-ed0d34e498f1\") " pod="openshift-ingress/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz" Jul 17 08:00:04.744649 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:04.744612 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9a11756-b5ff-4f48-bc78-ed0d34e498f1-istio-envoy\") pod \"e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz\" (UID: \"d9a11756-b5ff-4f48-bc78-ed0d34e498f1\") " pod="openshift-ingress/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz" Jul 17 08:00:04.744814 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:04.744662 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9a11756-b5ff-4f48-bc78-ed0d34e498f1-workload-certs\") pod \"e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz\" (UID: \"d9a11756-b5ff-4f48-bc78-ed0d34e498f1\") " pod="openshift-ingress/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz" Jul 17 08:00:04.744814 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:04.744691 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9a11756-b5ff-4f48-bc78-ed0d34e498f1-istiod-ca-cert\") pod \"e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz\" (UID: \"d9a11756-b5ff-4f48-bc78-ed0d34e498f1\") " pod="openshift-ingress/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz" Jul 17 08:00:04.744814 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:04.744741 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9a11756-b5ff-4f48-bc78-ed0d34e498f1-credential-socket\") pod \"e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz\" (UID: \"d9a11756-b5ff-4f48-bc78-ed0d34e498f1\") " pod="openshift-ingress/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz" Jul 17 08:00:04.744814 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:04.744768 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9a11756-b5ff-4f48-bc78-ed0d34e498f1-istio-podinfo\") pod \"e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz\" (UID: \"d9a11756-b5ff-4f48-bc78-ed0d34e498f1\") " pod="openshift-ingress/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz" Jul 17 08:00:04.744994 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:04.744811 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9a11756-b5ff-4f48-bc78-ed0d34e498f1-workload-socket\") pod \"e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz\" (UID: \"d9a11756-b5ff-4f48-bc78-ed0d34e498f1\") " pod="openshift-ingress/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz" Jul 17 08:00:04.744994 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:04.744850 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9a11756-b5ff-4f48-bc78-ed0d34e498f1-istio-data\") pod \"e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz\" (UID: \"d9a11756-b5ff-4f48-bc78-ed0d34e498f1\") " pod="openshift-ingress/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz" Jul 17 08:00:04.744994 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:04.744875 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9a11756-b5ff-4f48-bc78-ed0d34e498f1-istio-token\") pod \"e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz\" (UID: \"d9a11756-b5ff-4f48-bc78-ed0d34e498f1\") " pod="openshift-ingress/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz" Jul 17 08:00:04.744994 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:04.744978 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nhmn\" (UniqueName: \"kubernetes.io/projected/d9a11756-b5ff-4f48-bc78-ed0d34e498f1-kube-api-access-7nhmn\") pod \"e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz\" (UID: \"d9a11756-b5ff-4f48-bc78-ed0d34e498f1\") " pod="openshift-ingress/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz" Jul 17 08:00:04.745130 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:04.745040 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9a11756-b5ff-4f48-bc78-ed0d34e498f1-workload-certs\") pod \"e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz\" (UID: \"d9a11756-b5ff-4f48-bc78-ed0d34e498f1\") " pod="openshift-ingress/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz" Jul 17 08:00:04.745428 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:04.745400 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9a11756-b5ff-4f48-bc78-ed0d34e498f1-credential-socket\") pod \"e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz\" (UID: \"d9a11756-b5ff-4f48-bc78-ed0d34e498f1\") " pod="openshift-ingress/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz" Jul 17 08:00:04.745553 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:04.745457 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9a11756-b5ff-4f48-bc78-ed0d34e498f1-istio-data\") pod \"e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz\" (UID: \"d9a11756-b5ff-4f48-bc78-ed0d34e498f1\") " pod="openshift-ingress/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz" Jul 17 08:00:04.745553 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:04.745479 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9a11756-b5ff-4f48-bc78-ed0d34e498f1-istiod-ca-cert\") pod \"e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz\" (UID: \"d9a11756-b5ff-4f48-bc78-ed0d34e498f1\") " pod="openshift-ingress/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz" Jul 17 08:00:04.745553 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:04.745498 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9a11756-b5ff-4f48-bc78-ed0d34e498f1-workload-socket\") pod \"e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz\" (UID: \"d9a11756-b5ff-4f48-bc78-ed0d34e498f1\") " pod="openshift-ingress/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz" Jul 17 08:00:04.747132 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:04.747094 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9a11756-b5ff-4f48-bc78-ed0d34e498f1-istio-envoy\") pod \"e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz\" (UID: \"d9a11756-b5ff-4f48-bc78-ed0d34e498f1\") " pod="openshift-ingress/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz" Jul 17 08:00:04.747328 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:04.747308 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9a11756-b5ff-4f48-bc78-ed0d34e498f1-istio-podinfo\") pod \"e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz\" (UID: \"d9a11756-b5ff-4f48-bc78-ed0d34e498f1\") " pod="openshift-ingress/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz" Jul 17 08:00:04.757142 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:04.757117 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nhmn\" (UniqueName: \"kubernetes.io/projected/d9a11756-b5ff-4f48-bc78-ed0d34e498f1-kube-api-access-7nhmn\") pod \"e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz\" (UID: \"d9a11756-b5ff-4f48-bc78-ed0d34e498f1\") " pod="openshift-ingress/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz" Jul 17 08:00:04.757236 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:04.757117 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9a11756-b5ff-4f48-bc78-ed0d34e498f1-istio-token\") pod \"e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz\" (UID: \"d9a11756-b5ff-4f48-bc78-ed0d34e498f1\") " pod="openshift-ingress/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz" Jul 17 08:00:04.857410 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:04.857303 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz" Jul 17 08:00:05.023640 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:05.023615 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz"] Jul 17 08:00:05.025132 ip-10-0-138-233 kubenswrapper[2568]: W0717 08:00:05.025107 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9a11756_b5ff_4f48_bc78_ed0d34e498f1.slice/crio-530755d91fba92d938620f4fd04f9f67ca435d10230f8b284291c604bd294390 WatchSource:0}: Error finding container 530755d91fba92d938620f4fd04f9f67ca435d10230f8b284291c604bd294390: Status 404 returned error can't find the container with id 530755d91fba92d938620f4fd04f9f67ca435d10230f8b284291c604bd294390 Jul 17 08:00:05.027177 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:05.027138 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 08:00:05.027234 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:05.027208 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 08:00:05.027275 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:05.027241 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 08:00:05.919762 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:05.919729 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz" event={"ID":"d9a11756-b5ff-4f48-bc78-ed0d34e498f1","Type":"ContainerStarted","Data":"4e5c18e79ff35b8f188b846076d98768b4b379018ebd335c154320a1e9a8b9f8"} Jul 17 08:00:05.920165 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:05.919784 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz" event={"ID":"d9a11756-b5ff-4f48-bc78-ed0d34e498f1","Type":"ContainerStarted","Data":"530755d91fba92d938620f4fd04f9f67ca435d10230f8b284291c604bd294390"} Jul 17 08:00:05.944630 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:05.944580 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz" podStartSLOduration=1.944564417 podStartE2EDuration="1.944564417s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 08:00:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 08:00:05.943724179 +0000 UTC m=+2910.428413153" watchObservedRunningTime="2026-07-17 08:00:05.944564417 +0000 UTC m=+2910.429253391" Jul 17 08:00:06.216784 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:06.216697 2568 patch_prober.go:28] interesting pod/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 17 08:00:06.216784 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:06.216772 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv" podUID="5b7ae261-290d-48a8-9e0b-635e9abe3a4c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 17 08:00:06.858527 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:06.858491 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz" Jul 17 08:00:06.863314 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:06.863259 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz" Jul 17 08:00:06.925404 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:06.925374 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz" Jul 17 08:00:06.926253 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:06.926234 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz" Jul 17 08:00:08.855838 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:08.855778 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv" Jul 17 08:00:08.939452 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:08.939412 2568 generic.go:356] "Generic (PLEG): container finished" podID="5b7ae261-290d-48a8-9e0b-635e9abe3a4c" containerID="d6e74957bccf92a0cd870fbabc7b8a98bd9053a6637786dd49eb340445d8e0e0" exitCode=0 Jul 17 08:00:08.939977 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:08.939458 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv" event={"ID":"5b7ae261-290d-48a8-9e0b-635e9abe3a4c","Type":"ContainerDied","Data":"d6e74957bccf92a0cd870fbabc7b8a98bd9053a6637786dd49eb340445d8e0e0"} Jul 17 08:00:08.939977 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:08.939776 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv" event={"ID":"5b7ae261-290d-48a8-9e0b-635e9abe3a4c","Type":"ContainerDied","Data":"716df93ed843f23e31a5aaa6b54fda232029558940175038d0b671bfdb47192f"} Jul 17 08:00:08.939977 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:08.939482 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv" Jul 17 08:00:08.939977 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:08.939820 2568 scope.go:117] "RemoveContainer" containerID="d6e74957bccf92a0cd870fbabc7b8a98bd9053a6637786dd49eb340445d8e0e0" Jul 17 08:00:08.949524 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:08.949499 2568 scope.go:117] "RemoveContainer" containerID="8a23e70a610c973fe9025404d2148f0b268ee3b68849e56bb9c44cd7605740e8" Jul 17 08:00:08.958088 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:08.958074 2568 scope.go:117] "RemoveContainer" containerID="d6e74957bccf92a0cd870fbabc7b8a98bd9053a6637786dd49eb340445d8e0e0" Jul 17 08:00:08.958350 ip-10-0-138-233 kubenswrapper[2568]: E0717 08:00:08.958332 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6e74957bccf92a0cd870fbabc7b8a98bd9053a6637786dd49eb340445d8e0e0\": container with ID starting with d6e74957bccf92a0cd870fbabc7b8a98bd9053a6637786dd49eb340445d8e0e0 not found: ID does not exist" containerID="d6e74957bccf92a0cd870fbabc7b8a98bd9053a6637786dd49eb340445d8e0e0" Jul 17 08:00:08.958426 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:08.958357 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6e74957bccf92a0cd870fbabc7b8a98bd9053a6637786dd49eb340445d8e0e0"} err="failed to get container status \"d6e74957bccf92a0cd870fbabc7b8a98bd9053a6637786dd49eb340445d8e0e0\": rpc error: code = NotFound desc = could not find container \"d6e74957bccf92a0cd870fbabc7b8a98bd9053a6637786dd49eb340445d8e0e0\": container with ID starting with d6e74957bccf92a0cd870fbabc7b8a98bd9053a6637786dd49eb340445d8e0e0 not found: ID does not exist" Jul 17 08:00:08.958426 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:08.958372 2568 scope.go:117] "RemoveContainer" containerID="8a23e70a610c973fe9025404d2148f0b268ee3b68849e56bb9c44cd7605740e8" Jul 17 08:00:08.958621 ip-10-0-138-233 kubenswrapper[2568]: E0717 08:00:08.958603 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a23e70a610c973fe9025404d2148f0b268ee3b68849e56bb9c44cd7605740e8\": container with ID starting with 8a23e70a610c973fe9025404d2148f0b268ee3b68849e56bb9c44cd7605740e8 not found: ID does not exist" containerID="8a23e70a610c973fe9025404d2148f0b268ee3b68849e56bb9c44cd7605740e8" Jul 17 08:00:08.958679 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:08.958630 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a23e70a610c973fe9025404d2148f0b268ee3b68849e56bb9c44cd7605740e8"} err="failed to get container status \"8a23e70a610c973fe9025404d2148f0b268ee3b68849e56bb9c44cd7605740e8\": rpc error: code = NotFound desc = could not find container \"8a23e70a610c973fe9025404d2148f0b268ee3b68849e56bb9c44cd7605740e8\": container with ID starting with 8a23e70a610c973fe9025404d2148f0b268ee3b68849e56bb9c44cd7605740e8 not found: ID does not exist" Jul 17 08:00:08.983926 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:08.983902 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g8dvj\" (UniqueName: \"kubernetes.io/projected/5b7ae261-290d-48a8-9e0b-635e9abe3a4c-kube-api-access-g8dvj\") pod \"5b7ae261-290d-48a8-9e0b-635e9abe3a4c\" (UID: \"5b7ae261-290d-48a8-9e0b-635e9abe3a4c\") " Jul 17 08:00:08.984013 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:08.983932 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b7ae261-290d-48a8-9e0b-635e9abe3a4c-workload-certs\") pod \"5b7ae261-290d-48a8-9e0b-635e9abe3a4c\" (UID: \"5b7ae261-290d-48a8-9e0b-635e9abe3a4c\") " Jul 17 08:00:08.984013 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:08.983959 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5b7ae261-290d-48a8-9e0b-635e9abe3a4c-istio-podinfo\") pod \"5b7ae261-290d-48a8-9e0b-635e9abe3a4c\" (UID: \"5b7ae261-290d-48a8-9e0b-635e9abe3a4c\") " Jul 17 08:00:08.984013 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:08.983976 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b7ae261-290d-48a8-9e0b-635e9abe3a4c-istio-token\") pod \"5b7ae261-290d-48a8-9e0b-635e9abe3a4c\" (UID: \"5b7ae261-290d-48a8-9e0b-635e9abe3a4c\") " Jul 17 08:00:08.984013 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:08.983995 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b7ae261-290d-48a8-9e0b-635e9abe3a4c-credential-socket\") pod \"5b7ae261-290d-48a8-9e0b-635e9abe3a4c\" (UID: \"5b7ae261-290d-48a8-9e0b-635e9abe3a4c\") " Jul 17 08:00:08.984168 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:08.984054 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b7ae261-290d-48a8-9e0b-635e9abe3a4c-workload-socket\") pod \"5b7ae261-290d-48a8-9e0b-635e9abe3a4c\" (UID: \"5b7ae261-290d-48a8-9e0b-635e9abe3a4c\") " Jul 17 08:00:08.984168 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:08.984082 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5b7ae261-290d-48a8-9e0b-635e9abe3a4c-istiod-ca-cert\") pod \"5b7ae261-290d-48a8-9e0b-635e9abe3a4c\" (UID: \"5b7ae261-290d-48a8-9e0b-635e9abe3a4c\") " Jul 17 08:00:08.984168 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:08.984116 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5b7ae261-290d-48a8-9e0b-635e9abe3a4c-istio-envoy\") pod \"5b7ae261-290d-48a8-9e0b-635e9abe3a4c\" (UID: \"5b7ae261-290d-48a8-9e0b-635e9abe3a4c\") " Jul 17 08:00:08.984168 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:08.984144 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5b7ae261-290d-48a8-9e0b-635e9abe3a4c-istio-data\") pod \"5b7ae261-290d-48a8-9e0b-635e9abe3a4c\" (UID: \"5b7ae261-290d-48a8-9e0b-635e9abe3a4c\") " Jul 17 08:00:08.984393 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:08.984163 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b7ae261-290d-48a8-9e0b-635e9abe3a4c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5b7ae261-290d-48a8-9e0b-635e9abe3a4c" (UID: "5b7ae261-290d-48a8-9e0b-635e9abe3a4c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 08:00:08.984456 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:08.984373 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b7ae261-290d-48a8-9e0b-635e9abe3a4c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5b7ae261-290d-48a8-9e0b-635e9abe3a4c" (UID: "5b7ae261-290d-48a8-9e0b-635e9abe3a4c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 08:00:08.984612 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:08.984506 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b7ae261-290d-48a8-9e0b-635e9abe3a4c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5b7ae261-290d-48a8-9e0b-635e9abe3a4c" (UID: "5b7ae261-290d-48a8-9e0b-635e9abe3a4c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 08:00:08.984612 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:08.984537 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b7ae261-290d-48a8-9e0b-635e9abe3a4c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5b7ae261-290d-48a8-9e0b-635e9abe3a4c" (UID: "5b7ae261-290d-48a8-9e0b-635e9abe3a4c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 08:00:08.984761 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:08.984645 2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b7ae261-290d-48a8-9e0b-635e9abe3a4c-workload-socket\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 08:00:08.984761 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:08.984664 2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5b7ae261-290d-48a8-9e0b-635e9abe3a4c-istiod-ca-cert\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 08:00:08.984761 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:08.984677 2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b7ae261-290d-48a8-9e0b-635e9abe3a4c-workload-certs\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 08:00:08.984761 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:08.984690 2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b7ae261-290d-48a8-9e0b-635e9abe3a4c-credential-socket\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 08:00:08.985179 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:08.985153 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b7ae261-290d-48a8-9e0b-635e9abe3a4c-istio-data" (OuterVolumeSpecName: "istio-data") pod "5b7ae261-290d-48a8-9e0b-635e9abe3a4c" (UID: "5b7ae261-290d-48a8-9e0b-635e9abe3a4c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 08:00:08.986156 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:08.986131 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b7ae261-290d-48a8-9e0b-635e9abe3a4c-kube-api-access-g8dvj" (OuterVolumeSpecName: "kube-api-access-g8dvj") pod "5b7ae261-290d-48a8-9e0b-635e9abe3a4c" (UID: "5b7ae261-290d-48a8-9e0b-635e9abe3a4c"). InnerVolumeSpecName "kube-api-access-g8dvj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 08:00:08.986525 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:08.986506 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5b7ae261-290d-48a8-9e0b-635e9abe3a4c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5b7ae261-290d-48a8-9e0b-635e9abe3a4c" (UID: "5b7ae261-290d-48a8-9e0b-635e9abe3a4c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 17 08:00:08.986657 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:08.986633 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b7ae261-290d-48a8-9e0b-635e9abe3a4c-istio-token" (OuterVolumeSpecName: "istio-token") pod "5b7ae261-290d-48a8-9e0b-635e9abe3a4c" (UID: "5b7ae261-290d-48a8-9e0b-635e9abe3a4c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 08:00:08.986748 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:08.986681 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b7ae261-290d-48a8-9e0b-635e9abe3a4c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5b7ae261-290d-48a8-9e0b-635e9abe3a4c" (UID: "5b7ae261-290d-48a8-9e0b-635e9abe3a4c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 08:00:09.085393 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:09.085362 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g8dvj\" (UniqueName: \"kubernetes.io/projected/5b7ae261-290d-48a8-9e0b-635e9abe3a4c-kube-api-access-g8dvj\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 08:00:09.085393 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:09.085389 2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5b7ae261-290d-48a8-9e0b-635e9abe3a4c-istio-podinfo\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 08:00:09.085393 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:09.085399 2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b7ae261-290d-48a8-9e0b-635e9abe3a4c-istio-token\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 08:00:09.085656 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:09.085407 2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5b7ae261-290d-48a8-9e0b-635e9abe3a4c-istio-envoy\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 08:00:09.085656 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:09.085415 2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5b7ae261-290d-48a8-9e0b-635e9abe3a4c-istio-data\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 08:00:09.265265 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:09.265231 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv"] Jul 17 08:00:09.268817 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:09.268788 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-38e02d-openshift-default-55457d6b8f-r5jlv"] Jul 17 08:00:10.001007 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:10.000972 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b7ae261-290d-48a8-9e0b-635e9abe3a4c" path="/var/lib/kubelet/pods/5b7ae261-290d-48a8-9e0b-635e9abe3a4c/volumes" Jul 17 08:00:13.189164 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:13.189134 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6"] Jul 17 08:00:13.189797 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:13.189775 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b7ae261-290d-48a8-9e0b-635e9abe3a4c" containerName="istio-proxy" Jul 17 08:00:13.189797 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:13.189798 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b7ae261-290d-48a8-9e0b-635e9abe3a4c" containerName="istio-proxy" Jul 17 08:00:13.189962 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:13.189828 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b7ae261-290d-48a8-9e0b-635e9abe3a4c" containerName="istio-proxy" Jul 17 08:00:13.189962 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:13.189837 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b7ae261-290d-48a8-9e0b-635e9abe3a4c" containerName="istio-proxy" Jul 17 08:00:13.189962 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:13.189923 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d8af748-4c20-4576-87a2-29f50378e09b" containerName="istio-proxy" Jul 17 08:00:13.189962 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:13.189936 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b7ae261-290d-48a8-9e0b-635e9abe3a4c" containerName="istio-proxy" Jul 17 08:00:13.194429 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:13.194407 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6" Jul 17 08:00:13.197749 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:13.197716 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-04d1db-openshift-default-dockercfg-hp898\"" Jul 17 08:00:13.204906 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:13.204877 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6"] Jul 17 08:00:13.325018 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:13.324969 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ef884f3-a294-4171-87a5-7aa8e548fc04-istio-envoy\") pod \"e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6\" (UID: \"2ef884f3-a294-4171-87a5-7aa8e548fc04\") " pod="openshift-ingress/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6" Jul 17 08:00:13.325237 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:13.325071 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ef884f3-a294-4171-87a5-7aa8e548fc04-workload-socket\") pod \"e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6\" (UID: \"2ef884f3-a294-4171-87a5-7aa8e548fc04\") " pod="openshift-ingress/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6" Jul 17 08:00:13.325237 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:13.325135 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ef884f3-a294-4171-87a5-7aa8e548fc04-istio-token\") pod \"e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6\" (UID: \"2ef884f3-a294-4171-87a5-7aa8e548fc04\") " pod="openshift-ingress/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6" Jul 17 08:00:13.325237 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:13.325172 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ef884f3-a294-4171-87a5-7aa8e548fc04-credential-socket\") pod \"e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6\" (UID: \"2ef884f3-a294-4171-87a5-7aa8e548fc04\") " pod="openshift-ingress/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6" Jul 17 08:00:13.325237 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:13.325204 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlf8k\" (UniqueName: \"kubernetes.io/projected/2ef884f3-a294-4171-87a5-7aa8e548fc04-kube-api-access-tlf8k\") pod \"e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6\" (UID: \"2ef884f3-a294-4171-87a5-7aa8e548fc04\") " pod="openshift-ingress/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6" Jul 17 08:00:13.325477 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:13.325325 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ef884f3-a294-4171-87a5-7aa8e548fc04-istiod-ca-cert\") pod \"e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6\" (UID: \"2ef884f3-a294-4171-87a5-7aa8e548fc04\") " pod="openshift-ingress/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6" Jul 17 08:00:13.325477 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:13.325361 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ef884f3-a294-4171-87a5-7aa8e548fc04-workload-certs\") pod \"e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6\" (UID: \"2ef884f3-a294-4171-87a5-7aa8e548fc04\") " pod="openshift-ingress/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6" Jul 17 08:00:13.325477 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:13.325377 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ef884f3-a294-4171-87a5-7aa8e548fc04-istio-data\") pod \"e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6\" (UID: \"2ef884f3-a294-4171-87a5-7aa8e548fc04\") " pod="openshift-ingress/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6" Jul 17 08:00:13.325477 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:13.325406 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ef884f3-a294-4171-87a5-7aa8e548fc04-istio-podinfo\") pod \"e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6\" (UID: \"2ef884f3-a294-4171-87a5-7aa8e548fc04\") " pod="openshift-ingress/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6" Jul 17 08:00:13.426829 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:13.426783 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ef884f3-a294-4171-87a5-7aa8e548fc04-istiod-ca-cert\") pod \"e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6\" (UID: \"2ef884f3-a294-4171-87a5-7aa8e548fc04\") " pod="openshift-ingress/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6" Jul 17 08:00:13.427045 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:13.426865 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ef884f3-a294-4171-87a5-7aa8e548fc04-workload-certs\") pod \"e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6\" (UID: \"2ef884f3-a294-4171-87a5-7aa8e548fc04\") " pod="openshift-ingress/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6" Jul 17 08:00:13.427045 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:13.426891 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ef884f3-a294-4171-87a5-7aa8e548fc04-istio-data\") pod \"e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6\" (UID: \"2ef884f3-a294-4171-87a5-7aa8e548fc04\") " pod="openshift-ingress/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6" Jul 17 08:00:13.427045 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:13.426924 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ef884f3-a294-4171-87a5-7aa8e548fc04-istio-podinfo\") pod \"e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6\" (UID: \"2ef884f3-a294-4171-87a5-7aa8e548fc04\") " pod="openshift-ingress/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6" Jul 17 08:00:13.427045 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:13.426960 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ef884f3-a294-4171-87a5-7aa8e548fc04-istio-envoy\") pod \"e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6\" (UID: \"2ef884f3-a294-4171-87a5-7aa8e548fc04\") " pod="openshift-ingress/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6" Jul 17 08:00:13.427045 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:13.427023 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ef884f3-a294-4171-87a5-7aa8e548fc04-workload-socket\") pod \"e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6\" (UID: \"2ef884f3-a294-4171-87a5-7aa8e548fc04\") " pod="openshift-ingress/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6" Jul 17 08:00:13.427364 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:13.427073 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ef884f3-a294-4171-87a5-7aa8e548fc04-istio-token\") pod \"e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6\" (UID: \"2ef884f3-a294-4171-87a5-7aa8e548fc04\") " pod="openshift-ingress/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6" Jul 17 08:00:13.427364 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:13.427108 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ef884f3-a294-4171-87a5-7aa8e548fc04-credential-socket\") pod \"e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6\" (UID: \"2ef884f3-a294-4171-87a5-7aa8e548fc04\") " pod="openshift-ingress/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6" Jul 17 08:00:13.427364 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:13.427139 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlf8k\" (UniqueName: \"kubernetes.io/projected/2ef884f3-a294-4171-87a5-7aa8e548fc04-kube-api-access-tlf8k\") pod \"e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6\" (UID: \"2ef884f3-a294-4171-87a5-7aa8e548fc04\") " pod="openshift-ingress/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6" Jul 17 08:00:13.427545 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:13.427370 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ef884f3-a294-4171-87a5-7aa8e548fc04-workload-certs\") pod \"e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6\" (UID: \"2ef884f3-a294-4171-87a5-7aa8e548fc04\") " pod="openshift-ingress/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6" Jul 17 08:00:13.427545 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:13.427462 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ef884f3-a294-4171-87a5-7aa8e548fc04-workload-socket\") pod \"e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6\" (UID: \"2ef884f3-a294-4171-87a5-7aa8e548fc04\") " pod="openshift-ingress/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6" Jul 17 08:00:13.427760 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:13.427732 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ef884f3-a294-4171-87a5-7aa8e548fc04-istiod-ca-cert\") pod \"e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6\" (UID: \"2ef884f3-a294-4171-87a5-7aa8e548fc04\") " pod="openshift-ingress/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6" Jul 17 08:00:13.427854 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:13.427780 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ef884f3-a294-4171-87a5-7aa8e548fc04-credential-socket\") pod \"e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6\" (UID: \"2ef884f3-a294-4171-87a5-7aa8e548fc04\") " pod="openshift-ingress/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6" Jul 17 08:00:13.428134 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:13.428106 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ef884f3-a294-4171-87a5-7aa8e548fc04-istio-data\") pod \"e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6\" (UID: \"2ef884f3-a294-4171-87a5-7aa8e548fc04\") " pod="openshift-ingress/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6" Jul 17 08:00:13.429449 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:13.429423 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ef884f3-a294-4171-87a5-7aa8e548fc04-istio-envoy\") pod \"e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6\" (UID: \"2ef884f3-a294-4171-87a5-7aa8e548fc04\") " pod="openshift-ingress/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6" Jul 17 08:00:13.429663 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:13.429646 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ef884f3-a294-4171-87a5-7aa8e548fc04-istio-podinfo\") pod \"e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6\" (UID: \"2ef884f3-a294-4171-87a5-7aa8e548fc04\") " pod="openshift-ingress/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6" Jul 17 08:00:13.436152 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:13.436122 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ef884f3-a294-4171-87a5-7aa8e548fc04-istio-token\") pod \"e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6\" (UID: \"2ef884f3-a294-4171-87a5-7aa8e548fc04\") " pod="openshift-ingress/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6" Jul 17 08:00:13.436265 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:13.436249 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlf8k\" (UniqueName: \"kubernetes.io/projected/2ef884f3-a294-4171-87a5-7aa8e548fc04-kube-api-access-tlf8k\") pod \"e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6\" (UID: \"2ef884f3-a294-4171-87a5-7aa8e548fc04\") " pod="openshift-ingress/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6" Jul 17 08:00:13.506278 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:13.506175 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6" Jul 17 08:00:13.694660 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:13.692896 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6"] Jul 17 08:00:13.699231 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:13.699195 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 08:00:13.699368 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:13.699266 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 08:00:13.699368 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:13.699322 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 08:00:13.980506 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:13.980458 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6" event={"ID":"2ef884f3-a294-4171-87a5-7aa8e548fc04","Type":"ContainerStarted","Data":"83b91ef59982f7e9d281cb2ec1e1a52867c1afcac96dadc76ff3fb030c0e3ef4"} Jul 17 08:00:13.980720 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:13.980538 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6" event={"ID":"2ef884f3-a294-4171-87a5-7aa8e548fc04","Type":"ContainerStarted","Data":"c0e44048280b3b7a473c31727e6d67c2551d22a82f595c971691c27d80f16c4f"} Jul 17 08:00:14.029705 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:14.029655 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6" podStartSLOduration=1.029638466 podStartE2EDuration="1.029638466s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 08:00:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 08:00:14.028419964 +0000 UTC m=+2918.513108939" watchObservedRunningTime="2026-07-17 08:00:14.029638466 +0000 UTC m=+2918.514327439" Jul 17 08:00:14.506888 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:14.506858 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6" Jul 17 08:00:14.512440 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:14.512410 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6" Jul 17 08:00:14.990031 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:14.989858 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6" Jul 17 08:00:14.992725 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:14.992703 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6" Jul 17 08:00:21.026004 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:21.025967 2568 generic.go:356] "Generic (PLEG): container finished" podID="d9a11756-b5ff-4f48-bc78-ed0d34e498f1" containerID="4e5c18e79ff35b8f188b846076d98768b4b379018ebd335c154320a1e9a8b9f8" exitCode=0 Jul 17 08:00:21.026472 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:21.026040 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz" event={"ID":"d9a11756-b5ff-4f48-bc78-ed0d34e498f1","Type":"ContainerDied","Data":"4e5c18e79ff35b8f188b846076d98768b4b379018ebd335c154320a1e9a8b9f8"} Jul 17 08:00:21.026472 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:21.026447 2568 scope.go:117] "RemoveContainer" containerID="4e5c18e79ff35b8f188b846076d98768b4b379018ebd335c154320a1e9a8b9f8" Jul 17 08:00:21.027439 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:21.027407 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 08:00:21.027564 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:21.027488 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 08:00:21.027564 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:21.027536 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 08:00:21.925903 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:21.925865 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz" Jul 17 08:00:22.032845 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:22.032809 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz" event={"ID":"d9a11756-b5ff-4f48-bc78-ed0d34e498f1","Type":"ContainerStarted","Data":"b185a847dc9947f1b6846c1f50cb1f25cdd32ed1f065f74187107568752f4457"} Jul 17 08:00:22.858106 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:22.858066 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz" Jul 17 08:00:22.859090 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:22.859064 2568 patch_prober.go:28] interesting pod/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body= Jul 17 08:00:22.859186 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:22.859113 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz" podUID="d9a11756-b5ff-4f48-bc78-ed0d34e498f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" Jul 17 08:00:23.858463 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:23.858424 2568 patch_prober.go:28] interesting pod/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body= Jul 17 08:00:23.858850 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:23.858483 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz" podUID="d9a11756-b5ff-4f48-bc78-ed0d34e498f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" Jul 17 08:00:24.862170 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:24.862137 2568 patch_prober.go:28] interesting pod/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body= Jul 17 08:00:24.862566 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:24.862197 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz" podUID="d9a11756-b5ff-4f48-bc78-ed0d34e498f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" Jul 17 08:00:25.858524 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:25.858491 2568 patch_prober.go:28] interesting pod/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body= Jul 17 08:00:25.858714 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:25.858545 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz" podUID="d9a11756-b5ff-4f48-bc78-ed0d34e498f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" Jul 17 08:00:26.861611 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:26.861577 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz" Jul 17 08:00:26.862035 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:26.861846 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz" Jul 17 08:00:26.862486 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:26.862468 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz" Jul 17 08:00:30.866592 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:30.865840 2568 patch_prober.go:28] interesting pod/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": read tcp 10.132.0.2:51248->10.132.0.69:15021: read: connection reset by peer" start-of-body= Jul 17 08:00:30.866592 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:30.865922 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6" podUID="2ef884f3-a294-4171-87a5-7aa8e548fc04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": read tcp 10.132.0.2:51248->10.132.0.69:15021: read: connection reset by peer" Jul 17 08:00:30.908663 ip-10-0-138-233 kubenswrapper[2568]: E0717 08:00:30.908622 2568 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/system.slice/system-systemd\\\\x2dcoredump.slice/systemd-coredump@12-39457-0.service\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ef884f3_a294_4171_87a5_7aa8e548fc04.slice/crio-conmon-83b91ef59982f7e9d281cb2ec1e1a52867c1afcac96dadc76ff3fb030c0e3ef4.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ef884f3_a294_4171_87a5_7aa8e548fc04.slice/crio-83b91ef59982f7e9d281cb2ec1e1a52867c1afcac96dadc76ff3fb030c0e3ef4.scope\": RecentStats: unable to find data in memory cache]" Jul 17 08:00:31.084679 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:31.084585 2568 generic.go:356] "Generic (PLEG): container finished" podID="2ef884f3-a294-4171-87a5-7aa8e548fc04" containerID="83b91ef59982f7e9d281cb2ec1e1a52867c1afcac96dadc76ff3fb030c0e3ef4" exitCode=0 Jul 17 08:00:31.084679 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:31.084658 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6" event={"ID":"2ef884f3-a294-4171-87a5-7aa8e548fc04","Type":"ContainerDied","Data":"83b91ef59982f7e9d281cb2ec1e1a52867c1afcac96dadc76ff3fb030c0e3ef4"} Jul 17 08:00:31.085102 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:31.085088 2568 scope.go:117] "RemoveContainer" containerID="83b91ef59982f7e9d281cb2ec1e1a52867c1afcac96dadc76ff3fb030c0e3ef4" Jul 17 08:00:31.086095 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:31.086069 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 08:00:31.086177 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:31.086162 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 08:00:31.086220 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:31.086201 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 17 08:00:32.096870 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:32.096833 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6" event={"ID":"2ef884f3-a294-4171-87a5-7aa8e548fc04","Type":"ContainerStarted","Data":"a684f9f22d14d9f9c72dfe0c87bd118f6373bd175c7950557fceb93d8bd14872"} Jul 17 08:00:32.506640 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:32.506598 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6" Jul 17 08:00:32.506833 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:32.506802 2568 patch_prober.go:28] interesting pod/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body= Jul 17 08:00:32.506879 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:32.506848 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6" podUID="2ef884f3-a294-4171-87a5-7aa8e548fc04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" Jul 17 08:00:33.507174 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:33.507140 2568 patch_prober.go:28] interesting pod/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body= Jul 17 08:00:33.507608 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:33.507193 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6" podUID="2ef884f3-a294-4171-87a5-7aa8e548fc04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" Jul 17 08:00:34.507565 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:34.507526 2568 patch_prober.go:28] interesting pod/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body= Jul 17 08:00:34.507955 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:34.507582 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6" podUID="2ef884f3-a294-4171-87a5-7aa8e548fc04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" Jul 17 08:00:35.507394 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:35.507352 2568 patch_prober.go:28] interesting pod/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body= Jul 17 08:00:35.507615 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:35.507429 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6" podUID="2ef884f3-a294-4171-87a5-7aa8e548fc04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" Jul 17 08:00:36.510041 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:36.510010 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6" Jul 17 08:00:36.510511 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:36.510294 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6" Jul 17 08:00:36.511058 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:00:36.511041 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6" Jul 17 08:01:36.287416 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:01:36.287388 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-xr6pg_2ce00a54-15e9-42b7-b3a8-9e58d1101558/cluster-monitoring-operator/0.log" Jul 17 08:01:36.292875 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:01:36.292853 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-xpxv5_27538d1e-992b-4651-a2d2-7a7ecf442806/console-operator/1.log" Jul 17 08:01:36.296339 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:01:36.296323 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cpwz4_1c44a3ae-4c83-4203-8b0b-116d6618accf/ovn-acl-logging/0.log" Jul 17 08:01:36.304664 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:01:36.304643 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-xr6pg_2ce00a54-15e9-42b7-b3a8-9e58d1101558/cluster-monitoring-operator/0.log" Jul 17 08:01:36.309544 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:01:36.309526 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-xpxv5_27538d1e-992b-4651-a2d2-7a7ecf442806/console-operator/1.log" Jul 17 08:01:36.312755 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:01:36.312739 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cpwz4_1c44a3ae-4c83-4203-8b0b-116d6618accf/ovn-acl-logging/0.log" Jul 17 08:06:18.311979 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:18.311944 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7284-140537757629360"] Jul 17 08:06:18.312449 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:18.312415 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b7ae261-290d-48a8-9e0b-635e9abe3a4c" containerName="istio-proxy" Jul 17 08:06:18.315393 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:18.315374 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7284-140537757629360" Jul 17 08:06:18.318268 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:18.318246 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-7g5j2\"" Jul 17 08:06:18.321834 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:18.321814 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7284-140537757629360"] Jul 17 08:06:18.467026 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:18.466995 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h86w4\" (UniqueName: \"kubernetes.io/projected/f0e13197-e2c5-4d9d-aa8a-b5fd9726d2dd-kube-api-access-h86w4\") pod \"test-curl-7284-140537757629360\" (UID: \"f0e13197-e2c5-4d9d-aa8a-b5fd9726d2dd\") " pod="opendatahub/test-curl-7284-140537757629360" Jul 17 08:06:18.567909 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:18.567814 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h86w4\" (UniqueName: \"kubernetes.io/projected/f0e13197-e2c5-4d9d-aa8a-b5fd9726d2dd-kube-api-access-h86w4\") pod \"test-curl-7284-140537757629360\" (UID: \"f0e13197-e2c5-4d9d-aa8a-b5fd9726d2dd\") " pod="opendatahub/test-curl-7284-140537757629360" Jul 17 08:06:18.580327 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:18.580300 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h86w4\" (UniqueName: \"kubernetes.io/projected/f0e13197-e2c5-4d9d-aa8a-b5fd9726d2dd-kube-api-access-h86w4\") pod \"test-curl-7284-140537757629360\" (UID: \"f0e13197-e2c5-4d9d-aa8a-b5fd9726d2dd\") " pod="opendatahub/test-curl-7284-140537757629360" Jul 17 08:06:18.627333 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:18.627309 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7284-140537757629360" Jul 17 08:06:18.764422 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:18.764396 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7284-140537757629360"] Jul 17 08:06:18.765456 ip-10-0-138-233 kubenswrapper[2568]: W0717 08:06:18.765424 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf0e13197_e2c5_4d9d_aa8a_b5fd9726d2dd.slice/crio-aa72045fe35ea815a01220048cebe3450b6492aef18964d4c19bbeb0c6988dc4 WatchSource:0}: Error finding container aa72045fe35ea815a01220048cebe3450b6492aef18964d4c19bbeb0c6988dc4: Status 404 returned error can't find the container with id aa72045fe35ea815a01220048cebe3450b6492aef18964d4c19bbeb0c6988dc4 Jul 17 08:06:18.766802 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:18.766785 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 17 08:06:19.037525 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:19.037490 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7284-140537757629360" event={"ID":"f0e13197-e2c5-4d9d-aa8a-b5fd9726d2dd","Type":"ContainerStarted","Data":"aa72045fe35ea815a01220048cebe3450b6492aef18964d4c19bbeb0c6988dc4"} Jul 17 08:06:20.044701 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:20.044664 2568 generic.go:356] "Generic (PLEG): container finished" podID="f0e13197-e2c5-4d9d-aa8a-b5fd9726d2dd" containerID="080965d79417e5193393de0207ad9a018a6a08ca2f855a14b397d423a6f2feb2" exitCode=0 Jul 17 08:06:20.045091 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:20.044707 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7284-140537757629360" event={"ID":"f0e13197-e2c5-4d9d-aa8a-b5fd9726d2dd","Type":"ContainerDied","Data":"080965d79417e5193393de0207ad9a018a6a08ca2f855a14b397d423a6f2feb2"} Jul 17 08:06:21.189717 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:21.189692 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7284-140537757629360" Jul 17 08:06:21.236392 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:21.236361 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7284-140537757629360_f0e13197-e2c5-4d9d-aa8a-b5fd9726d2dd/test-curl-7284-140537757629360/0.log" Jul 17 08:06:21.266633 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:21.266592 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7284-140537757629360"] Jul 17 08:06:21.274659 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:21.274625 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7284-140537757629360"] Jul 17 08:06:21.294088 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:21.294059 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h86w4\" (UniqueName: \"kubernetes.io/projected/f0e13197-e2c5-4d9d-aa8a-b5fd9726d2dd-kube-api-access-h86w4\") pod \"f0e13197-e2c5-4d9d-aa8a-b5fd9726d2dd\" (UID: \"f0e13197-e2c5-4d9d-aa8a-b5fd9726d2dd\") " Jul 17 08:06:21.296178 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:21.296146 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0e13197-e2c5-4d9d-aa8a-b5fd9726d2dd-kube-api-access-h86w4" (OuterVolumeSpecName: "kube-api-access-h86w4") pod "f0e13197-e2c5-4d9d-aa8a-b5fd9726d2dd" (UID: "f0e13197-e2c5-4d9d-aa8a-b5fd9726d2dd"). InnerVolumeSpecName "kube-api-access-h86w4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 08:06:21.395537 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:21.395502 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h86w4\" (UniqueName: \"kubernetes.io/projected/f0e13197-e2c5-4d9d-aa8a-b5fd9726d2dd-kube-api-access-h86w4\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 08:06:21.408546 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:21.408511 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7284-140537757655472"] Jul 17 08:06:21.408909 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:21.408895 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0e13197-e2c5-4d9d-aa8a-b5fd9726d2dd" containerName="test-curl-7284-140537757629360" Jul 17 08:06:21.408971 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:21.408910 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0e13197-e2c5-4d9d-aa8a-b5fd9726d2dd" containerName="test-curl-7284-140537757629360" Jul 17 08:06:21.409025 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:21.408980 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0e13197-e2c5-4d9d-aa8a-b5fd9726d2dd" containerName="test-curl-7284-140537757629360" Jul 17 08:06:21.412032 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:21.412012 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7284-140537757655472" Jul 17 08:06:21.420731 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:21.420709 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7284-140537757655472"] Jul 17 08:06:21.496052 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:21.496017 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lshtf\" (UniqueName: \"kubernetes.io/projected/cfbbf7a8-aacf-4a63-a9c2-627fa31de591-kube-api-access-lshtf\") pod \"test-curl-7284-140537757655472\" (UID: \"cfbbf7a8-aacf-4a63-a9c2-627fa31de591\") " pod="opendatahub/test-curl-7284-140537757655472" Jul 17 08:06:21.596842 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:21.596757 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lshtf\" (UniqueName: \"kubernetes.io/projected/cfbbf7a8-aacf-4a63-a9c2-627fa31de591-kube-api-access-lshtf\") pod \"test-curl-7284-140537757655472\" (UID: \"cfbbf7a8-aacf-4a63-a9c2-627fa31de591\") " pod="opendatahub/test-curl-7284-140537757655472" Jul 17 08:06:21.606105 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:21.606073 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lshtf\" (UniqueName: \"kubernetes.io/projected/cfbbf7a8-aacf-4a63-a9c2-627fa31de591-kube-api-access-lshtf\") pod \"test-curl-7284-140537757655472\" (UID: \"cfbbf7a8-aacf-4a63-a9c2-627fa31de591\") " pod="opendatahub/test-curl-7284-140537757655472" Jul 17 08:06:21.721890 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:21.721863 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7284-140537757655472" Jul 17 08:06:21.861350 ip-10-0-138-233 kubenswrapper[2568]: W0717 08:06:21.861318 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcfbbf7a8_aacf_4a63_a9c2_627fa31de591.slice/crio-9c1f03c0642f48a3dc41c941ec08731ec264e7fdfb7e238f3520d471c12697a0 WatchSource:0}: Error finding container 9c1f03c0642f48a3dc41c941ec08731ec264e7fdfb7e238f3520d471c12697a0: Status 404 returned error can't find the container with id 9c1f03c0642f48a3dc41c941ec08731ec264e7fdfb7e238f3520d471c12697a0 Jul 17 08:06:21.861674 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:21.861652 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7284-140537757655472"] Jul 17 08:06:22.004752 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:22.004717 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0e13197-e2c5-4d9d-aa8a-b5fd9726d2dd" path="/var/lib/kubelet/pods/f0e13197-e2c5-4d9d-aa8a-b5fd9726d2dd/volumes" Jul 17 08:06:22.056577 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:22.056549 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7284-140537757655472" event={"ID":"cfbbf7a8-aacf-4a63-a9c2-627fa31de591","Type":"ContainerStarted","Data":"9c1f03c0642f48a3dc41c941ec08731ec264e7fdfb7e238f3520d471c12697a0"} Jul 17 08:06:22.057695 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:22.057673 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7284-140537757629360" Jul 17 08:06:22.057825 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:22.057685 2568 scope.go:117] "RemoveContainer" containerID="080965d79417e5193393de0207ad9a018a6a08ca2f855a14b397d423a6f2feb2" Jul 17 08:06:23.065487 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:23.065442 2568 generic.go:356] "Generic (PLEG): container finished" podID="cfbbf7a8-aacf-4a63-a9c2-627fa31de591" containerID="d1a260edf03f9791cad8cb0dfd09ffc761d16d6b9a7204d3ed0f2280517d6e88" exitCode=0 Jul 17 08:06:23.065910 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:23.065533 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7284-140537757655472" event={"ID":"cfbbf7a8-aacf-4a63-a9c2-627fa31de591","Type":"ContainerDied","Data":"d1a260edf03f9791cad8cb0dfd09ffc761d16d6b9a7204d3ed0f2280517d6e88"} Jul 17 08:06:24.211200 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:24.211171 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7284-140537757655472" Jul 17 08:06:24.235338 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:24.235315 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7284-140537757655472_cfbbf7a8-aacf-4a63-a9c2-627fa31de591/test-curl-7284-140537757655472/0.log" Jul 17 08:06:24.266199 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:24.266172 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7284-140537757655472"] Jul 17 08:06:24.271406 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:24.271381 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7284-140537757655472"] Jul 17 08:06:24.321972 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:24.321895 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lshtf\" (UniqueName: \"kubernetes.io/projected/cfbbf7a8-aacf-4a63-a9c2-627fa31de591-kube-api-access-lshtf\") pod \"cfbbf7a8-aacf-4a63-a9c2-627fa31de591\" (UID: \"cfbbf7a8-aacf-4a63-a9c2-627fa31de591\") " Jul 17 08:06:24.323758 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:24.323727 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfbbf7a8-aacf-4a63-a9c2-627fa31de591-kube-api-access-lshtf" (OuterVolumeSpecName: "kube-api-access-lshtf") pod "cfbbf7a8-aacf-4a63-a9c2-627fa31de591" (UID: "cfbbf7a8-aacf-4a63-a9c2-627fa31de591"). InnerVolumeSpecName "kube-api-access-lshtf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 08:06:24.423442 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:24.423413 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lshtf\" (UniqueName: \"kubernetes.io/projected/cfbbf7a8-aacf-4a63-a9c2-627fa31de591-kube-api-access-lshtf\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 08:06:24.468957 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:24.468927 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7284-140537757710352"] Jul 17 08:06:24.469411 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:24.469393 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfbbf7a8-aacf-4a63-a9c2-627fa31de591" containerName="test-curl-7284-140537757655472" Jul 17 08:06:24.469516 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:24.469412 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfbbf7a8-aacf-4a63-a9c2-627fa31de591" containerName="test-curl-7284-140537757655472" Jul 17 08:06:24.469567 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:24.469522 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfbbf7a8-aacf-4a63-a9c2-627fa31de591" containerName="test-curl-7284-140537757655472" Jul 17 08:06:24.472525 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:24.472506 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7284-140537757710352" Jul 17 08:06:24.478318 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:24.478295 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7284-140537757710352"] Jul 17 08:06:24.625730 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:24.625707 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbsbl\" (UniqueName: \"kubernetes.io/projected/73bbd8c8-c4c1-4dcb-a95e-9238f1c0a760-kube-api-access-bbsbl\") pod \"test-curl-7284-140537757710352\" (UID: \"73bbd8c8-c4c1-4dcb-a95e-9238f1c0a760\") " pod="opendatahub/test-curl-7284-140537757710352" Jul 17 08:06:24.727039 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:24.727005 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbsbl\" (UniqueName: \"kubernetes.io/projected/73bbd8c8-c4c1-4dcb-a95e-9238f1c0a760-kube-api-access-bbsbl\") pod \"test-curl-7284-140537757710352\" (UID: \"73bbd8c8-c4c1-4dcb-a95e-9238f1c0a760\") " pod="opendatahub/test-curl-7284-140537757710352" Jul 17 08:06:24.737555 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:24.737524 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbsbl\" (UniqueName: \"kubernetes.io/projected/73bbd8c8-c4c1-4dcb-a95e-9238f1c0a760-kube-api-access-bbsbl\") pod \"test-curl-7284-140537757710352\" (UID: \"73bbd8c8-c4c1-4dcb-a95e-9238f1c0a760\") " pod="opendatahub/test-curl-7284-140537757710352" Jul 17 08:06:24.783820 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:24.783798 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7284-140537757710352" Jul 17 08:06:24.920237 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:24.920212 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7284-140537757710352"] Jul 17 08:06:24.921556 ip-10-0-138-233 kubenswrapper[2568]: W0717 08:06:24.921524 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod73bbd8c8_c4c1_4dcb_a95e_9238f1c0a760.slice/crio-e82685c11ce4de37186582f6afaedc2cc186023fb08c4b8758633e76df500ead WatchSource:0}: Error finding container e82685c11ce4de37186582f6afaedc2cc186023fb08c4b8758633e76df500ead: Status 404 returned error can't find the container with id e82685c11ce4de37186582f6afaedc2cc186023fb08c4b8758633e76df500ead Jul 17 08:06:25.077820 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:25.077790 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7284-140537757655472" Jul 17 08:06:25.077967 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:25.077823 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9c1f03c0642f48a3dc41c941ec08731ec264e7fdfb7e238f3520d471c12697a0" Jul 17 08:06:25.078986 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:25.078963 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7284-140537757710352" event={"ID":"73bbd8c8-c4c1-4dcb-a95e-9238f1c0a760","Type":"ContainerStarted","Data":"e82685c11ce4de37186582f6afaedc2cc186023fb08c4b8758633e76df500ead"} Jul 17 08:06:25.088564 ip-10-0-138-233 kubenswrapper[2568]: E0717 08:06:25.088538 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7284-140537757655472\" is forbidden: User \"system:node:ip-10-0-138-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-233.ec2.internal' and this object" podUID="cfbbf7a8-aacf-4a63-a9c2-627fa31de591" pod="opendatahub/test-curl-7284-140537757655472" Jul 17 08:06:26.003143 ip-10-0-138-233 kubenswrapper[2568]: E0717 08:06:26.003099 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7284-140537757655472\" is forbidden: User \"system:node:ip-10-0-138-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-233.ec2.internal' and this object" podUID="cfbbf7a8-aacf-4a63-a9c2-627fa31de591" pod="opendatahub/test-curl-7284-140537757655472" Jul 17 08:06:26.007402 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:26.007378 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cfbbf7a8-aacf-4a63-a9c2-627fa31de591" path="/var/lib/kubelet/pods/cfbbf7a8-aacf-4a63-a9c2-627fa31de591/volumes" Jul 17 08:06:26.086061 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:26.086028 2568 generic.go:356] "Generic (PLEG): container finished" podID="73bbd8c8-c4c1-4dcb-a95e-9238f1c0a760" containerID="1605b7e2da803df6fd9e8ef7e4e6dbdded9cb5417469fbd5bab6ee15e1d60cb9" exitCode=0 Jul 17 08:06:26.086224 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:26.086113 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7284-140537757710352" event={"ID":"73bbd8c8-c4c1-4dcb-a95e-9238f1c0a760","Type":"ContainerDied","Data":"1605b7e2da803df6fd9e8ef7e4e6dbdded9cb5417469fbd5bab6ee15e1d60cb9"} Jul 17 08:06:27.228016 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:27.227994 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7284-140537757710352" Jul 17 08:06:27.251208 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:27.251184 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7284-140537757710352_73bbd8c8-c4c1-4dcb-a95e-9238f1c0a760/test-curl-7284-140537757710352/0.log" Jul 17 08:06:27.278241 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:27.278215 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7284-140537757710352"] Jul 17 08:06:27.282126 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:27.282104 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7284-140537757710352"] Jul 17 08:06:27.350185 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:27.350103 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bbsbl\" (UniqueName: \"kubernetes.io/projected/73bbd8c8-c4c1-4dcb-a95e-9238f1c0a760-kube-api-access-bbsbl\") pod \"73bbd8c8-c4c1-4dcb-a95e-9238f1c0a760\" (UID: \"73bbd8c8-c4c1-4dcb-a95e-9238f1c0a760\") " Jul 17 08:06:27.352203 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:27.352173 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73bbd8c8-c4c1-4dcb-a95e-9238f1c0a760-kube-api-access-bbsbl" (OuterVolumeSpecName: "kube-api-access-bbsbl") pod "73bbd8c8-c4c1-4dcb-a95e-9238f1c0a760" (UID: "73bbd8c8-c4c1-4dcb-a95e-9238f1c0a760"). InnerVolumeSpecName "kube-api-access-bbsbl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 08:06:27.420637 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:27.420607 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7284-140537757710352"] Jul 17 08:06:27.421019 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:27.421006 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73bbd8c8-c4c1-4dcb-a95e-9238f1c0a760" containerName="test-curl-7284-140537757710352" Jul 17 08:06:27.421068 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:27.421021 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="73bbd8c8-c4c1-4dcb-a95e-9238f1c0a760" containerName="test-curl-7284-140537757710352" Jul 17 08:06:27.421107 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:27.421086 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="73bbd8c8-c4c1-4dcb-a95e-9238f1c0a760" containerName="test-curl-7284-140537757710352" Jul 17 08:06:27.424016 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:27.424001 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7284-140537757710352" Jul 17 08:06:27.432523 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:27.432501 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7284-140537757710352"] Jul 17 08:06:27.451318 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:27.451278 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bbsbl\" (UniqueName: \"kubernetes.io/projected/73bbd8c8-c4c1-4dcb-a95e-9238f1c0a760-kube-api-access-bbsbl\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 08:06:27.552354 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:27.552329 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlhbl\" (UniqueName: \"kubernetes.io/projected/9f6d8a7d-8f11-45aa-bf3f-d0ae7057c5fb-kube-api-access-zlhbl\") pod \"test-curl-7284-140537757710352\" (UID: \"9f6d8a7d-8f11-45aa-bf3f-d0ae7057c5fb\") " pod="opendatahub/test-curl-7284-140537757710352" Jul 17 08:06:27.653057 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:27.653029 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlhbl\" (UniqueName: \"kubernetes.io/projected/9f6d8a7d-8f11-45aa-bf3f-d0ae7057c5fb-kube-api-access-zlhbl\") pod \"test-curl-7284-140537757710352\" (UID: \"9f6d8a7d-8f11-45aa-bf3f-d0ae7057c5fb\") " pod="opendatahub/test-curl-7284-140537757710352" Jul 17 08:06:27.662673 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:27.662645 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlhbl\" (UniqueName: \"kubernetes.io/projected/9f6d8a7d-8f11-45aa-bf3f-d0ae7057c5fb-kube-api-access-zlhbl\") pod \"test-curl-7284-140537757710352\" (UID: \"9f6d8a7d-8f11-45aa-bf3f-d0ae7057c5fb\") " pod="opendatahub/test-curl-7284-140537757710352" Jul 17 08:06:27.733736 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:27.733707 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7284-140537757710352" Jul 17 08:06:27.869578 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:27.869548 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7284-140537757710352"] Jul 17 08:06:27.871077 ip-10-0-138-233 kubenswrapper[2568]: W0717 08:06:27.871049 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9f6d8a7d_8f11_45aa_bf3f_d0ae7057c5fb.slice/crio-a4f982362a7e3a2dcf6a97311212371f98bd7268b47bcda5d9e7b73404d9ce0d WatchSource:0}: Error finding container a4f982362a7e3a2dcf6a97311212371f98bd7268b47bcda5d9e7b73404d9ce0d: Status 404 returned error can't find the container with id a4f982362a7e3a2dcf6a97311212371f98bd7268b47bcda5d9e7b73404d9ce0d Jul 17 08:06:27.999859 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:27.999827 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73bbd8c8-c4c1-4dcb-a95e-9238f1c0a760" path="/var/lib/kubelet/pods/73bbd8c8-c4c1-4dcb-a95e-9238f1c0a760/volumes" Jul 17 08:06:28.098075 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:28.098040 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7284-140537757710352" event={"ID":"9f6d8a7d-8f11-45aa-bf3f-d0ae7057c5fb","Type":"ContainerStarted","Data":"a4f982362a7e3a2dcf6a97311212371f98bd7268b47bcda5d9e7b73404d9ce0d"} Jul 17 08:06:28.099379 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:28.099358 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7284-140537757710352" Jul 17 08:06:28.099484 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:28.099366 2568 scope.go:117] "RemoveContainer" containerID="1605b7e2da803df6fd9e8ef7e4e6dbdded9cb5417469fbd5bab6ee15e1d60cb9" Jul 17 08:06:29.106552 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:29.106515 2568 generic.go:356] "Generic (PLEG): container finished" podID="9f6d8a7d-8f11-45aa-bf3f-d0ae7057c5fb" containerID="18d47d649e001c580079731637589e5cda035b15a23b51455f84c1a5e4b0ab1c" exitCode=0 Jul 17 08:06:29.106942 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:29.106603 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7284-140537757710352" event={"ID":"9f6d8a7d-8f11-45aa-bf3f-d0ae7057c5fb","Type":"ContainerDied","Data":"18d47d649e001c580079731637589e5cda035b15a23b51455f84c1a5e4b0ab1c"} Jul 17 08:06:30.252671 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:30.252643 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7284-140537757710352" Jul 17 08:06:30.276219 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:30.276190 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7284-140537757710352_9f6d8a7d-8f11-45aa-bf3f-d0ae7057c5fb/test-curl-7284-140537757710352/0.log" Jul 17 08:06:30.305390 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:30.305364 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7284-140537757710352"] Jul 17 08:06:30.309525 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:30.309504 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7284-140537757710352"] Jul 17 08:06:30.379644 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:30.379574 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zlhbl\" (UniqueName: \"kubernetes.io/projected/9f6d8a7d-8f11-45aa-bf3f-d0ae7057c5fb-kube-api-access-zlhbl\") pod \"9f6d8a7d-8f11-45aa-bf3f-d0ae7057c5fb\" (UID: \"9f6d8a7d-8f11-45aa-bf3f-d0ae7057c5fb\") " Jul 17 08:06:30.381647 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:30.381620 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f6d8a7d-8f11-45aa-bf3f-d0ae7057c5fb-kube-api-access-zlhbl" (OuterVolumeSpecName: "kube-api-access-zlhbl") pod "9f6d8a7d-8f11-45aa-bf3f-d0ae7057c5fb" (UID: "9f6d8a7d-8f11-45aa-bf3f-d0ae7057c5fb"). InnerVolumeSpecName "kube-api-access-zlhbl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 08:06:30.448394 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:30.448358 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7284-140537757711360"] Jul 17 08:06:30.448767 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:30.448754 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f6d8a7d-8f11-45aa-bf3f-d0ae7057c5fb" containerName="test-curl-7284-140537757710352" Jul 17 08:06:30.448809 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:30.448768 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f6d8a7d-8f11-45aa-bf3f-d0ae7057c5fb" containerName="test-curl-7284-140537757710352" Jul 17 08:06:30.448844 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:30.448832 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f6d8a7d-8f11-45aa-bf3f-d0ae7057c5fb" containerName="test-curl-7284-140537757710352" Jul 17 08:06:30.451669 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:30.451653 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7284-140537757711360" Jul 17 08:06:30.461307 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:30.461272 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7284-140537757711360"] Jul 17 08:06:30.480972 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:30.480946 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zlhbl\" (UniqueName: \"kubernetes.io/projected/9f6d8a7d-8f11-45aa-bf3f-d0ae7057c5fb-kube-api-access-zlhbl\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 08:06:30.581809 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:30.581775 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fllr6\" (UniqueName: \"kubernetes.io/projected/15d835f5-cd32-4d1f-97fb-a12f14b37d0b-kube-api-access-fllr6\") pod \"test-curl-7284-140537757711360\" (UID: \"15d835f5-cd32-4d1f-97fb-a12f14b37d0b\") " pod="opendatahub/test-curl-7284-140537757711360" Jul 17 08:06:30.683190 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:30.683094 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fllr6\" (UniqueName: \"kubernetes.io/projected/15d835f5-cd32-4d1f-97fb-a12f14b37d0b-kube-api-access-fllr6\") pod \"test-curl-7284-140537757711360\" (UID: \"15d835f5-cd32-4d1f-97fb-a12f14b37d0b\") " pod="opendatahub/test-curl-7284-140537757711360" Jul 17 08:06:30.692836 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:30.692813 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fllr6\" (UniqueName: \"kubernetes.io/projected/15d835f5-cd32-4d1f-97fb-a12f14b37d0b-kube-api-access-fllr6\") pod \"test-curl-7284-140537757711360\" (UID: \"15d835f5-cd32-4d1f-97fb-a12f14b37d0b\") " pod="opendatahub/test-curl-7284-140537757711360" Jul 17 08:06:30.762729 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:30.762702 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7284-140537757711360" Jul 17 08:06:30.929825 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:30.929790 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7284-140537757711360"] Jul 17 08:06:30.932908 ip-10-0-138-233 kubenswrapper[2568]: W0717 08:06:30.932880 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod15d835f5_cd32_4d1f_97fb_a12f14b37d0b.slice/crio-eaf7fb557a7d8a80fecee7692ce4dccdeacc7a8900e98501ca3503dccf149848 WatchSource:0}: Error finding container eaf7fb557a7d8a80fecee7692ce4dccdeacc7a8900e98501ca3503dccf149848: Status 404 returned error can't find the container with id eaf7fb557a7d8a80fecee7692ce4dccdeacc7a8900e98501ca3503dccf149848 Jul 17 08:06:31.119186 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:31.119155 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7284-140537757710352" Jul 17 08:06:31.119407 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:31.119192 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a4f982362a7e3a2dcf6a97311212371f98bd7268b47bcda5d9e7b73404d9ce0d" Jul 17 08:06:31.120429 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:31.120393 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7284-140537757711360" event={"ID":"15d835f5-cd32-4d1f-97fb-a12f14b37d0b","Type":"ContainerStarted","Data":"eaf7fb557a7d8a80fecee7692ce4dccdeacc7a8900e98501ca3503dccf149848"} Jul 17 08:06:31.130152 ip-10-0-138-233 kubenswrapper[2568]: E0717 08:06:31.130122 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7284-140537757710352\" is forbidden: User \"system:node:ip-10-0-138-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-233.ec2.internal' and this object" podUID="9f6d8a7d-8f11-45aa-bf3f-d0ae7057c5fb" pod="opendatahub/test-curl-7284-140537757710352" Jul 17 08:06:32.001819 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:32.001783 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f6d8a7d-8f11-45aa-bf3f-d0ae7057c5fb" path="/var/lib/kubelet/pods/9f6d8a7d-8f11-45aa-bf3f-d0ae7057c5fb/volumes" Jul 17 08:06:32.128428 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:32.128393 2568 generic.go:356] "Generic (PLEG): container finished" podID="15d835f5-cd32-4d1f-97fb-a12f14b37d0b" containerID="386f6a35d981da5dfa5b126a372b83b57709f6a9d2ee2c9bb15da91cc68b3eb0" exitCode=0 Jul 17 08:06:32.128595 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:32.128445 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7284-140537757711360" event={"ID":"15d835f5-cd32-4d1f-97fb-a12f14b37d0b","Type":"ContainerDied","Data":"386f6a35d981da5dfa5b126a372b83b57709f6a9d2ee2c9bb15da91cc68b3eb0"} Jul 17 08:06:33.273866 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:33.273841 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7284-140537757711360" Jul 17 08:06:33.295432 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:33.295405 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7284-140537757711360_15d835f5-cd32-4d1f-97fb-a12f14b37d0b/test-curl-7284-140537757711360/0.log" Jul 17 08:06:33.308713 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:33.308684 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fllr6\" (UniqueName: \"kubernetes.io/projected/15d835f5-cd32-4d1f-97fb-a12f14b37d0b-kube-api-access-fllr6\") pod \"15d835f5-cd32-4d1f-97fb-a12f14b37d0b\" (UID: \"15d835f5-cd32-4d1f-97fb-a12f14b37d0b\") " Jul 17 08:06:33.310768 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:33.310740 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15d835f5-cd32-4d1f-97fb-a12f14b37d0b-kube-api-access-fllr6" (OuterVolumeSpecName: "kube-api-access-fllr6") pod "15d835f5-cd32-4d1f-97fb-a12f14b37d0b" (UID: "15d835f5-cd32-4d1f-97fb-a12f14b37d0b"). InnerVolumeSpecName "kube-api-access-fllr6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 08:06:33.323019 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:33.322998 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7284-140537757711360"] Jul 17 08:06:33.327412 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:33.327391 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7284-140537757711360"] Jul 17 08:06:33.409740 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:33.409674 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fllr6\" (UniqueName: \"kubernetes.io/projected/15d835f5-cd32-4d1f-97fb-a12f14b37d0b-kube-api-access-fllr6\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 08:06:33.458546 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:33.458518 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7284-140537757711360"] Jul 17 08:06:33.458944 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:33.458929 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15d835f5-cd32-4d1f-97fb-a12f14b37d0b" containerName="test-curl-7284-140537757711360" Jul 17 08:06:33.459001 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:33.458946 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="15d835f5-cd32-4d1f-97fb-a12f14b37d0b" containerName="test-curl-7284-140537757711360" Jul 17 08:06:33.459037 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:33.459024 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="15d835f5-cd32-4d1f-97fb-a12f14b37d0b" containerName="test-curl-7284-140537757711360" Jul 17 08:06:33.461975 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:33.461958 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7284-140537757711360" Jul 17 08:06:33.466917 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:33.466896 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7284-140537757711360"] Jul 17 08:06:33.510368 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:33.510341 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phh7f\" (UniqueName: \"kubernetes.io/projected/99d407bd-1ebc-4dcf-91c3-2d728e6fe729-kube-api-access-phh7f\") pod \"test-curl-7284-140537757711360\" (UID: \"99d407bd-1ebc-4dcf-91c3-2d728e6fe729\") " pod="opendatahub/test-curl-7284-140537757711360" Jul 17 08:06:33.611000 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:33.610965 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phh7f\" (UniqueName: \"kubernetes.io/projected/99d407bd-1ebc-4dcf-91c3-2d728e6fe729-kube-api-access-phh7f\") pod \"test-curl-7284-140537757711360\" (UID: \"99d407bd-1ebc-4dcf-91c3-2d728e6fe729\") " pod="opendatahub/test-curl-7284-140537757711360" Jul 17 08:06:33.620797 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:33.620772 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-phh7f\" (UniqueName: \"kubernetes.io/projected/99d407bd-1ebc-4dcf-91c3-2d728e6fe729-kube-api-access-phh7f\") pod \"test-curl-7284-140537757711360\" (UID: \"99d407bd-1ebc-4dcf-91c3-2d728e6fe729\") " pod="opendatahub/test-curl-7284-140537757711360" Jul 17 08:06:33.772796 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:33.772719 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7284-140537757711360" Jul 17 08:06:33.912020 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:33.911995 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7284-140537757711360"] Jul 17 08:06:33.913338 ip-10-0-138-233 kubenswrapper[2568]: W0717 08:06:33.913313 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod99d407bd_1ebc_4dcf_91c3_2d728e6fe729.slice/crio-1ac45a65f2910c61055d196c0c70bd2be7e83e9921d0d47e967b1750788ed209 WatchSource:0}: Error finding container 1ac45a65f2910c61055d196c0c70bd2be7e83e9921d0d47e967b1750788ed209: Status 404 returned error can't find the container with id 1ac45a65f2910c61055d196c0c70bd2be7e83e9921d0d47e967b1750788ed209 Jul 17 08:06:34.000621 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:34.000593 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15d835f5-cd32-4d1f-97fb-a12f14b37d0b" path="/var/lib/kubelet/pods/15d835f5-cd32-4d1f-97fb-a12f14b37d0b/volumes" Jul 17 08:06:34.147143 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:34.147104 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7284-140537757711360" event={"ID":"99d407bd-1ebc-4dcf-91c3-2d728e6fe729","Type":"ContainerStarted","Data":"1ac45a65f2910c61055d196c0c70bd2be7e83e9921d0d47e967b1750788ed209"} Jul 17 08:06:34.148606 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:34.148562 2568 scope.go:117] "RemoveContainer" containerID="386f6a35d981da5dfa5b126a372b83b57709f6a9d2ee2c9bb15da91cc68b3eb0" Jul 17 08:06:34.148760 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:34.148588 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7284-140537757711360" Jul 17 08:06:35.156272 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:35.156237 2568 generic.go:356] "Generic (PLEG): container finished" podID="99d407bd-1ebc-4dcf-91c3-2d728e6fe729" containerID="1b9ddbe30059cce548335f68e4b0c1492fd101bda4ac68c9fcbc1fa54d6500e1" exitCode=0 Jul 17 08:06:35.156707 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:35.156323 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7284-140537757711360" event={"ID":"99d407bd-1ebc-4dcf-91c3-2d728e6fe729","Type":"ContainerDied","Data":"1b9ddbe30059cce548335f68e4b0c1492fd101bda4ac68c9fcbc1fa54d6500e1"} Jul 17 08:06:36.293468 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:36.293444 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7284-140537757711360" Jul 17 08:06:36.316399 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:36.316022 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7284-140537757711360_99d407bd-1ebc-4dcf-91c3-2d728e6fe729/test-curl-7284-140537757711360/0.log" Jul 17 08:06:36.334029 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:36.334006 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-xr6pg_2ce00a54-15e9-42b7-b3a8-9e58d1101558/cluster-monitoring-operator/0.log" Jul 17 08:06:36.335190 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:36.335163 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-phh7f\" (UniqueName: \"kubernetes.io/projected/99d407bd-1ebc-4dcf-91c3-2d728e6fe729-kube-api-access-phh7f\") pod \"99d407bd-1ebc-4dcf-91c3-2d728e6fe729\" (UID: \"99d407bd-1ebc-4dcf-91c3-2d728e6fe729\") " Jul 17 08:06:36.337646 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:36.337620 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99d407bd-1ebc-4dcf-91c3-2d728e6fe729-kube-api-access-phh7f" (OuterVolumeSpecName: "kube-api-access-phh7f") pod "99d407bd-1ebc-4dcf-91c3-2d728e6fe729" (UID: "99d407bd-1ebc-4dcf-91c3-2d728e6fe729"). InnerVolumeSpecName "kube-api-access-phh7f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 08:06:36.341897 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:36.341875 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-xpxv5_27538d1e-992b-4651-a2d2-7a7ecf442806/console-operator/1.log" Jul 17 08:06:36.343798 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:36.343772 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7284-140537757711360"] Jul 17 08:06:36.346852 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:36.346831 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cpwz4_1c44a3ae-4c83-4203-8b0b-116d6618accf/ovn-acl-logging/0.log" Jul 17 08:06:36.347910 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:36.347890 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7284-140537757711360"] Jul 17 08:06:36.353644 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:36.353627 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-xr6pg_2ce00a54-15e9-42b7-b3a8-9e58d1101558/cluster-monitoring-operator/0.log" Jul 17 08:06:36.358094 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:36.358077 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-xpxv5_27538d1e-992b-4651-a2d2-7a7ecf442806/console-operator/1.log" Jul 17 08:06:36.361235 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:36.361217 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cpwz4_1c44a3ae-4c83-4203-8b0b-116d6618accf/ovn-acl-logging/0.log" Jul 17 08:06:36.436271 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:36.436197 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-phh7f\" (UniqueName: \"kubernetes.io/projected/99d407bd-1ebc-4dcf-91c3-2d728e6fe729-kube-api-access-phh7f\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 08:06:37.170058 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:37.170032 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1ac45a65f2910c61055d196c0c70bd2be7e83e9921d0d47e967b1750788ed209" Jul 17 08:06:37.170058 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:37.170038 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7284-140537757711360" Jul 17 08:06:37.181379 ip-10-0-138-233 kubenswrapper[2568]: E0717 08:06:37.181348 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7284-140537757711360\" is forbidden: User \"system:node:ip-10-0-138-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-233.ec2.internal' and this object" podUID="99d407bd-1ebc-4dcf-91c3-2d728e6fe729" pod="opendatahub/test-curl-7284-140537757711360" Jul 17 08:06:38.000183 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:06:38.000151 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99d407bd-1ebc-4dcf-91c3-2d728e6fe729" path="/var/lib/kubelet/pods/99d407bd-1ebc-4dcf-91c3-2d728e6fe729/volumes" Jul 17 08:07:02.419488 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:02.419400 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz"] Jul 17 08:07:02.420051 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:02.419596 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz" podUID="d9a11756-b5ff-4f48-bc78-ed0d34e498f1" containerName="istio-proxy" containerID="cri-o://b185a847dc9947f1b6846c1f50cb1f25cdd32ed1f065f74187107568752f4457" gracePeriod=30 Jul 17 08:07:07.672308 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:07.672262 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz" Jul 17 08:07:07.838060 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:07.837983 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9a11756-b5ff-4f48-bc78-ed0d34e498f1-credential-socket\") pod \"d9a11756-b5ff-4f48-bc78-ed0d34e498f1\" (UID: \"d9a11756-b5ff-4f48-bc78-ed0d34e498f1\") " Jul 17 08:07:07.838060 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:07.838047 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9a11756-b5ff-4f48-bc78-ed0d34e498f1-workload-socket\") pod \"d9a11756-b5ff-4f48-bc78-ed0d34e498f1\" (UID: \"d9a11756-b5ff-4f48-bc78-ed0d34e498f1\") " Jul 17 08:07:07.838310 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:07.838081 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9a11756-b5ff-4f48-bc78-ed0d34e498f1-istio-data\") pod \"d9a11756-b5ff-4f48-bc78-ed0d34e498f1\" (UID: \"d9a11756-b5ff-4f48-bc78-ed0d34e498f1\") " Jul 17 08:07:07.838310 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:07.838123 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9a11756-b5ff-4f48-bc78-ed0d34e498f1-istiod-ca-cert\") pod \"d9a11756-b5ff-4f48-bc78-ed0d34e498f1\" (UID: \"d9a11756-b5ff-4f48-bc78-ed0d34e498f1\") " Jul 17 08:07:07.838310 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:07.838152 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9a11756-b5ff-4f48-bc78-ed0d34e498f1-istio-envoy\") pod \"d9a11756-b5ff-4f48-bc78-ed0d34e498f1\" (UID: \"d9a11756-b5ff-4f48-bc78-ed0d34e498f1\") " Jul 17 08:07:07.838310 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:07.838170 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9a11756-b5ff-4f48-bc78-ed0d34e498f1-istio-token\") pod \"d9a11756-b5ff-4f48-bc78-ed0d34e498f1\" (UID: \"d9a11756-b5ff-4f48-bc78-ed0d34e498f1\") " Jul 17 08:07:07.838540 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:07.838316 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9a11756-b5ff-4f48-bc78-ed0d34e498f1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d9a11756-b5ff-4f48-bc78-ed0d34e498f1" (UID: "d9a11756-b5ff-4f48-bc78-ed0d34e498f1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 08:07:07.838540 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:07.838400 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7nhmn\" (UniqueName: \"kubernetes.io/projected/d9a11756-b5ff-4f48-bc78-ed0d34e498f1-kube-api-access-7nhmn\") pod \"d9a11756-b5ff-4f48-bc78-ed0d34e498f1\" (UID: \"d9a11756-b5ff-4f48-bc78-ed0d34e498f1\") " Jul 17 08:07:07.838540 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:07.838423 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9a11756-b5ff-4f48-bc78-ed0d34e498f1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d9a11756-b5ff-4f48-bc78-ed0d34e498f1" (UID: "d9a11756-b5ff-4f48-bc78-ed0d34e498f1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 08:07:07.838540 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:07.838432 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9a11756-b5ff-4f48-bc78-ed0d34e498f1-istio-podinfo\") pod \"d9a11756-b5ff-4f48-bc78-ed0d34e498f1\" (UID: \"d9a11756-b5ff-4f48-bc78-ed0d34e498f1\") " Jul 17 08:07:07.838540 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:07.838504 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9a11756-b5ff-4f48-bc78-ed0d34e498f1-workload-certs\") pod \"d9a11756-b5ff-4f48-bc78-ed0d34e498f1\" (UID: \"d9a11756-b5ff-4f48-bc78-ed0d34e498f1\") " Jul 17 08:07:07.838830 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:07.838555 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9a11756-b5ff-4f48-bc78-ed0d34e498f1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d9a11756-b5ff-4f48-bc78-ed0d34e498f1" (UID: "d9a11756-b5ff-4f48-bc78-ed0d34e498f1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 08:07:07.838915 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:07.838894 2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9a11756-b5ff-4f48-bc78-ed0d34e498f1-workload-socket\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 08:07:07.838967 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:07.838929 2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9a11756-b5ff-4f48-bc78-ed0d34e498f1-istiod-ca-cert\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 08:07:07.838967 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:07.838945 2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9a11756-b5ff-4f48-bc78-ed0d34e498f1-credential-socket\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 08:07:07.838967 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:07.838894 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9a11756-b5ff-4f48-bc78-ed0d34e498f1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d9a11756-b5ff-4f48-bc78-ed0d34e498f1" (UID: "d9a11756-b5ff-4f48-bc78-ed0d34e498f1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 08:07:07.839087 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:07.838982 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9a11756-b5ff-4f48-bc78-ed0d34e498f1-istio-data" (OuterVolumeSpecName: "istio-data") pod "d9a11756-b5ff-4f48-bc78-ed0d34e498f1" (UID: "d9a11756-b5ff-4f48-bc78-ed0d34e498f1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 08:07:07.840506 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:07.840481 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d9a11756-b5ff-4f48-bc78-ed0d34e498f1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d9a11756-b5ff-4f48-bc78-ed0d34e498f1" (UID: "d9a11756-b5ff-4f48-bc78-ed0d34e498f1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 17 08:07:07.840601 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:07.840565 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9a11756-b5ff-4f48-bc78-ed0d34e498f1-kube-api-access-7nhmn" (OuterVolumeSpecName: "kube-api-access-7nhmn") pod "d9a11756-b5ff-4f48-bc78-ed0d34e498f1" (UID: "d9a11756-b5ff-4f48-bc78-ed0d34e498f1"). InnerVolumeSpecName "kube-api-access-7nhmn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 08:07:07.840766 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:07.840741 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9a11756-b5ff-4f48-bc78-ed0d34e498f1-istio-token" (OuterVolumeSpecName: "istio-token") pod "d9a11756-b5ff-4f48-bc78-ed0d34e498f1" (UID: "d9a11756-b5ff-4f48-bc78-ed0d34e498f1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 08:07:07.840766 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:07.840752 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9a11756-b5ff-4f48-bc78-ed0d34e498f1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d9a11756-b5ff-4f48-bc78-ed0d34e498f1" (UID: "d9a11756-b5ff-4f48-bc78-ed0d34e498f1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 08:07:07.940336 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:07.940302 2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9a11756-b5ff-4f48-bc78-ed0d34e498f1-istio-data\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 08:07:07.940336 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:07.940330 2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9a11756-b5ff-4f48-bc78-ed0d34e498f1-istio-envoy\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 08:07:07.940336 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:07.940339 2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9a11756-b5ff-4f48-bc78-ed0d34e498f1-istio-token\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 08:07:07.940562 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:07.940348 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7nhmn\" (UniqueName: \"kubernetes.io/projected/d9a11756-b5ff-4f48-bc78-ed0d34e498f1-kube-api-access-7nhmn\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 08:07:07.940562 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:07.940357 2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9a11756-b5ff-4f48-bc78-ed0d34e498f1-istio-podinfo\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 08:07:07.940562 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:07.940366 2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9a11756-b5ff-4f48-bc78-ed0d34e498f1-workload-certs\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 08:07:08.346151 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:08.346113 2568 generic.go:356] "Generic (PLEG): container finished" podID="d9a11756-b5ff-4f48-bc78-ed0d34e498f1" containerID="b185a847dc9947f1b6846c1f50cb1f25cdd32ed1f065f74187107568752f4457" exitCode=0 Jul 17 08:07:08.346357 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:08.346159 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz" Jul 17 08:07:08.346357 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:08.346194 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz" event={"ID":"d9a11756-b5ff-4f48-bc78-ed0d34e498f1","Type":"ContainerDied","Data":"b185a847dc9947f1b6846c1f50cb1f25cdd32ed1f065f74187107568752f4457"} Jul 17 08:07:08.346357 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:08.346249 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz" event={"ID":"d9a11756-b5ff-4f48-bc78-ed0d34e498f1","Type":"ContainerDied","Data":"530755d91fba92d938620f4fd04f9f67ca435d10230f8b284291c604bd294390"} Jul 17 08:07:08.346357 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:08.346271 2568 scope.go:117] "RemoveContainer" containerID="b185a847dc9947f1b6846c1f50cb1f25cdd32ed1f065f74187107568752f4457" Jul 17 08:07:08.355861 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:08.355843 2568 scope.go:117] "RemoveContainer" containerID="4e5c18e79ff35b8f188b846076d98768b4b379018ebd335c154320a1e9a8b9f8" Jul 17 08:07:08.364035 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:08.364021 2568 scope.go:117] "RemoveContainer" containerID="b185a847dc9947f1b6846c1f50cb1f25cdd32ed1f065f74187107568752f4457" Jul 17 08:07:08.364351 ip-10-0-138-233 kubenswrapper[2568]: E0717 08:07:08.364307 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b185a847dc9947f1b6846c1f50cb1f25cdd32ed1f065f74187107568752f4457\": container with ID starting with b185a847dc9947f1b6846c1f50cb1f25cdd32ed1f065f74187107568752f4457 not found: ID does not exist" containerID="b185a847dc9947f1b6846c1f50cb1f25cdd32ed1f065f74187107568752f4457" Jul 17 08:07:08.364452 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:08.364361 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b185a847dc9947f1b6846c1f50cb1f25cdd32ed1f065f74187107568752f4457"} err="failed to get container status \"b185a847dc9947f1b6846c1f50cb1f25cdd32ed1f065f74187107568752f4457\": rpc error: code = NotFound desc = could not find container \"b185a847dc9947f1b6846c1f50cb1f25cdd32ed1f065f74187107568752f4457\": container with ID starting with b185a847dc9947f1b6846c1f50cb1f25cdd32ed1f065f74187107568752f4457 not found: ID does not exist" Jul 17 08:07:08.364566 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:08.364434 2568 scope.go:117] "RemoveContainer" containerID="4e5c18e79ff35b8f188b846076d98768b4b379018ebd335c154320a1e9a8b9f8" Jul 17 08:07:08.364878 ip-10-0-138-233 kubenswrapper[2568]: E0717 08:07:08.364783 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e5c18e79ff35b8f188b846076d98768b4b379018ebd335c154320a1e9a8b9f8\": container with ID starting with 4e5c18e79ff35b8f188b846076d98768b4b379018ebd335c154320a1e9a8b9f8 not found: ID does not exist" containerID="4e5c18e79ff35b8f188b846076d98768b4b379018ebd335c154320a1e9a8b9f8" Jul 17 08:07:08.364878 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:08.364816 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e5c18e79ff35b8f188b846076d98768b4b379018ebd335c154320a1e9a8b9f8"} err="failed to get container status \"4e5c18e79ff35b8f188b846076d98768b4b379018ebd335c154320a1e9a8b9f8\": rpc error: code = NotFound desc = could not find container \"4e5c18e79ff35b8f188b846076d98768b4b379018ebd335c154320a1e9a8b9f8\": container with ID starting with 4e5c18e79ff35b8f188b846076d98768b4b379018ebd335c154320a1e9a8b9f8 not found: ID does not exist" Jul 17 08:07:08.367304 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:08.367273 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz"] Jul 17 08:07:08.380124 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:08.380104 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-d805bd-openshift-default-9b475f8dc-6jqfz"] Jul 17 08:07:10.000622 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:10.000588 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9a11756-b5ff-4f48-bc78-ed0d34e498f1" path="/var/lib/kubelet/pods/d9a11756-b5ff-4f48-bc78-ed0d34e498f1/volumes" Jul 17 08:07:28.672639 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:28.672606 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6"] Jul 17 08:07:28.673212 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:28.672906 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6" podUID="2ef884f3-a294-4171-87a5-7aa8e548fc04" containerName="istio-proxy" containerID="cri-o://a684f9f22d14d9f9c72dfe0c87bd118f6373bd175c7950557fceb93d8bd14872" gracePeriod=30 Jul 17 08:07:33.936334 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:33.936263 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6" Jul 17 08:07:34.071077 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:34.071047 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ef884f3-a294-4171-87a5-7aa8e548fc04-istio-podinfo\") pod \"2ef884f3-a294-4171-87a5-7aa8e548fc04\" (UID: \"2ef884f3-a294-4171-87a5-7aa8e548fc04\") " Jul 17 08:07:34.071269 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:34.071090 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ef884f3-a294-4171-87a5-7aa8e548fc04-istio-token\") pod \"2ef884f3-a294-4171-87a5-7aa8e548fc04\" (UID: \"2ef884f3-a294-4171-87a5-7aa8e548fc04\") " Jul 17 08:07:34.071269 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:34.071115 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ef884f3-a294-4171-87a5-7aa8e548fc04-credential-socket\") pod \"2ef884f3-a294-4171-87a5-7aa8e548fc04\" (UID: \"2ef884f3-a294-4171-87a5-7aa8e548fc04\") " Jul 17 08:07:34.071269 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:34.071141 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ef884f3-a294-4171-87a5-7aa8e548fc04-istiod-ca-cert\") pod \"2ef884f3-a294-4171-87a5-7aa8e548fc04\" (UID: \"2ef884f3-a294-4171-87a5-7aa8e548fc04\") " Jul 17 08:07:34.071269 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:34.071155 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ef884f3-a294-4171-87a5-7aa8e548fc04-workload-socket\") pod \"2ef884f3-a294-4171-87a5-7aa8e548fc04\" (UID: \"2ef884f3-a294-4171-87a5-7aa8e548fc04\") " Jul 17 08:07:34.071269 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:34.071194 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ef884f3-a294-4171-87a5-7aa8e548fc04-workload-certs\") pod \"2ef884f3-a294-4171-87a5-7aa8e548fc04\" (UID: \"2ef884f3-a294-4171-87a5-7aa8e548fc04\") " Jul 17 08:07:34.071269 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:34.071255 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ef884f3-a294-4171-87a5-7aa8e548fc04-istio-envoy\") pod \"2ef884f3-a294-4171-87a5-7aa8e548fc04\" (UID: \"2ef884f3-a294-4171-87a5-7aa8e548fc04\") " Jul 17 08:07:34.071660 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:34.071318 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ef884f3-a294-4171-87a5-7aa8e548fc04-istio-data\") pod \"2ef884f3-a294-4171-87a5-7aa8e548fc04\" (UID: \"2ef884f3-a294-4171-87a5-7aa8e548fc04\") " Jul 17 08:07:34.071660 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:34.071377 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tlf8k\" (UniqueName: \"kubernetes.io/projected/2ef884f3-a294-4171-87a5-7aa8e548fc04-kube-api-access-tlf8k\") pod \"2ef884f3-a294-4171-87a5-7aa8e548fc04\" (UID: \"2ef884f3-a294-4171-87a5-7aa8e548fc04\") " Jul 17 08:07:34.071660 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:34.071522 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ef884f3-a294-4171-87a5-7aa8e548fc04-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2ef884f3-a294-4171-87a5-7aa8e548fc04" (UID: "2ef884f3-a294-4171-87a5-7aa8e548fc04"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 08:07:34.071660 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:34.071543 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ef884f3-a294-4171-87a5-7aa8e548fc04-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2ef884f3-a294-4171-87a5-7aa8e548fc04" (UID: "2ef884f3-a294-4171-87a5-7aa8e548fc04"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 08:07:34.071878 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:34.071690 2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ef884f3-a294-4171-87a5-7aa8e548fc04-credential-socket\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 08:07:34.071878 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:34.071707 2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ef884f3-a294-4171-87a5-7aa8e548fc04-workload-certs\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 08:07:34.071878 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:34.071721 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ef884f3-a294-4171-87a5-7aa8e548fc04-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2ef884f3-a294-4171-87a5-7aa8e548fc04" (UID: "2ef884f3-a294-4171-87a5-7aa8e548fc04"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 08:07:34.072092 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:34.071941 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ef884f3-a294-4171-87a5-7aa8e548fc04-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2ef884f3-a294-4171-87a5-7aa8e548fc04" (UID: "2ef884f3-a294-4171-87a5-7aa8e548fc04"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 08:07:34.072275 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:34.072246 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ef884f3-a294-4171-87a5-7aa8e548fc04-istio-data" (OuterVolumeSpecName: "istio-data") pod "2ef884f3-a294-4171-87a5-7aa8e548fc04" (UID: "2ef884f3-a294-4171-87a5-7aa8e548fc04"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 08:07:34.073441 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:34.073422 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ef884f3-a294-4171-87a5-7aa8e548fc04-istio-token" (OuterVolumeSpecName: "istio-token") pod "2ef884f3-a294-4171-87a5-7aa8e548fc04" (UID: "2ef884f3-a294-4171-87a5-7aa8e548fc04"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 08:07:34.073569 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:34.073441 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2ef884f3-a294-4171-87a5-7aa8e548fc04-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2ef884f3-a294-4171-87a5-7aa8e548fc04" (UID: "2ef884f3-a294-4171-87a5-7aa8e548fc04"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 17 08:07:34.073569 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:34.073474 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ef884f3-a294-4171-87a5-7aa8e548fc04-kube-api-access-tlf8k" (OuterVolumeSpecName: "kube-api-access-tlf8k") pod "2ef884f3-a294-4171-87a5-7aa8e548fc04" (UID: "2ef884f3-a294-4171-87a5-7aa8e548fc04"). InnerVolumeSpecName "kube-api-access-tlf8k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 08:07:34.073755 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:34.073741 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ef884f3-a294-4171-87a5-7aa8e548fc04-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2ef884f3-a294-4171-87a5-7aa8e548fc04" (UID: "2ef884f3-a294-4171-87a5-7aa8e548fc04"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 08:07:34.172452 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:34.172418 2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ef884f3-a294-4171-87a5-7aa8e548fc04-istio-envoy\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 08:07:34.172452 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:34.172446 2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ef884f3-a294-4171-87a5-7aa8e548fc04-istio-data\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 08:07:34.172452 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:34.172455 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tlf8k\" (UniqueName: \"kubernetes.io/projected/2ef884f3-a294-4171-87a5-7aa8e548fc04-kube-api-access-tlf8k\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 08:07:34.172697 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:34.172466 2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ef884f3-a294-4171-87a5-7aa8e548fc04-istio-podinfo\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 08:07:34.172697 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:34.172474 2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ef884f3-a294-4171-87a5-7aa8e548fc04-istio-token\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 08:07:34.172697 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:34.172482 2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ef884f3-a294-4171-87a5-7aa8e548fc04-istiod-ca-cert\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 08:07:34.172697 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:34.172491 2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ef884f3-a294-4171-87a5-7aa8e548fc04-workload-socket\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\"" Jul 17 08:07:34.502378 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:34.502343 2568 generic.go:356] "Generic (PLEG): container finished" podID="2ef884f3-a294-4171-87a5-7aa8e548fc04" containerID="a684f9f22d14d9f9c72dfe0c87bd118f6373bd175c7950557fceb93d8bd14872" exitCode=0 Jul 17 08:07:34.502548 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:34.502389 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6" Jul 17 08:07:34.502548 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:34.502423 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6" event={"ID":"2ef884f3-a294-4171-87a5-7aa8e548fc04","Type":"ContainerDied","Data":"a684f9f22d14d9f9c72dfe0c87bd118f6373bd175c7950557fceb93d8bd14872"} Jul 17 08:07:34.502548 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:34.502480 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6" event={"ID":"2ef884f3-a294-4171-87a5-7aa8e548fc04","Type":"ContainerDied","Data":"c0e44048280b3b7a473c31727e6d67c2551d22a82f595c971691c27d80f16c4f"} Jul 17 08:07:34.502548 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:34.502503 2568 scope.go:117] "RemoveContainer" containerID="a684f9f22d14d9f9c72dfe0c87bd118f6373bd175c7950557fceb93d8bd14872" Jul 17 08:07:34.512573 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:34.512533 2568 scope.go:117] "RemoveContainer" containerID="83b91ef59982f7e9d281cb2ec1e1a52867c1afcac96dadc76ff3fb030c0e3ef4" Jul 17 08:07:34.521574 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:34.521554 2568 scope.go:117] "RemoveContainer" containerID="a684f9f22d14d9f9c72dfe0c87bd118f6373bd175c7950557fceb93d8bd14872" Jul 17 08:07:34.521818 ip-10-0-138-233 kubenswrapper[2568]: E0717 08:07:34.521797 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a684f9f22d14d9f9c72dfe0c87bd118f6373bd175c7950557fceb93d8bd14872\": container with ID starting with a684f9f22d14d9f9c72dfe0c87bd118f6373bd175c7950557fceb93d8bd14872 not found: ID does not exist" containerID="a684f9f22d14d9f9c72dfe0c87bd118f6373bd175c7950557fceb93d8bd14872" Jul 17 08:07:34.521859 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:34.521826 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a684f9f22d14d9f9c72dfe0c87bd118f6373bd175c7950557fceb93d8bd14872"} err="failed to get container status \"a684f9f22d14d9f9c72dfe0c87bd118f6373bd175c7950557fceb93d8bd14872\": rpc error: code = NotFound desc = could not find container \"a684f9f22d14d9f9c72dfe0c87bd118f6373bd175c7950557fceb93d8bd14872\": container with ID starting with a684f9f22d14d9f9c72dfe0c87bd118f6373bd175c7950557fceb93d8bd14872 not found: ID does not exist" Jul 17 08:07:34.521859 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:34.521843 2568 scope.go:117] "RemoveContainer" containerID="83b91ef59982f7e9d281cb2ec1e1a52867c1afcac96dadc76ff3fb030c0e3ef4" Jul 17 08:07:34.522060 ip-10-0-138-233 kubenswrapper[2568]: E0717 08:07:34.522044 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83b91ef59982f7e9d281cb2ec1e1a52867c1afcac96dadc76ff3fb030c0e3ef4\": container with ID starting with 83b91ef59982f7e9d281cb2ec1e1a52867c1afcac96dadc76ff3fb030c0e3ef4 not found: ID does not exist" containerID="83b91ef59982f7e9d281cb2ec1e1a52867c1afcac96dadc76ff3fb030c0e3ef4" Jul 17 08:07:34.522105 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:34.522066 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83b91ef59982f7e9d281cb2ec1e1a52867c1afcac96dadc76ff3fb030c0e3ef4"} err="failed to get container status \"83b91ef59982f7e9d281cb2ec1e1a52867c1afcac96dadc76ff3fb030c0e3ef4\": rpc error: code = NotFound desc = could not find container \"83b91ef59982f7e9d281cb2ec1e1a52867c1afcac96dadc76ff3fb030c0e3ef4\": container with ID starting with 83b91ef59982f7e9d281cb2ec1e1a52867c1afcac96dadc76ff3fb030c0e3ef4 not found: ID does not exist" Jul 17 08:07:34.526713 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:34.526690 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6"] Jul 17 08:07:34.533261 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:34.533240 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-04d1db-openshift-default-64b769ff55-kwdr6"] Jul 17 08:07:35.505146 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:35.505109 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-nznpn_563e3f68-d742-4e2b-8cab-03385da4780c/manager/0.log" Jul 17 08:07:35.642366 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:35.642333 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-kskbn_92188035-d0d9-4c04-b496-0d53544cde37/manager/0.log" Jul 17 08:07:35.779891 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:35.779814 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-579d999db4-94vrg_222416ad-c37f-4cc0-946f-7c11384a9f91/manager/0.log" Jul 17 08:07:35.913931 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:35.913902 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-smgzc_14f64e3c-e222-4720-b723-358eeed0c402/server/0.log" Jul 17 08:07:36.000969 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:36.000942 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ef884f3-a294-4171-87a5-7aa8e548fc04" path="/var/lib/kubelet/pods/2ef884f3-a294-4171-87a5-7aa8e548fc04/volumes" Jul 17 08:07:36.046807 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:36.046743 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-7kqk8_8dc175df-3be1-4f44-aaed-fe6ee8e60771/manager/0.log" Jul 17 08:07:36.176044 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:36.176016 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59ffcf4dc8-56dhq_6a4b5d36-d807-4d62-8aaf-6432767c269c/copy-manifests/0.log" Jul 17 08:07:36.190313 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:36.190276 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59ffcf4dc8-56dhq_6a4b5d36-d807-4d62-8aaf-6432767c269c/manager/0.log" Jul 17 08:07:36.864987 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:36.864961 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-7fffcb47b4-j9l8r_1f61d02c-327d-4010-97ec-737f365bd17a/maas-api/0.log" Jul 17 08:07:38.466341 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:38.466309 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-smfbz_b2a05aa6-6f0b-43f0-aa68-79996fd6de35/postgres/0.log" Jul 17 08:07:39.319138 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:39.319106 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599rfrw_eb41a9f6-2e49-48dc-aee9-737d3b981c3e/util/0.log" Jul 17 08:07:39.326594 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:39.326566 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599rfrw_eb41a9f6-2e49-48dc-aee9-737d3b981c3e/pull/0.log" Jul 17 08:07:39.333747 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:39.333725 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599rfrw_eb41a9f6-2e49-48dc-aee9-737d3b981c3e/extract/0.log" Jul 17 08:07:39.460985 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:39.460959 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066lbr_1448f170-548f-4e5d-a909-d1c97039252a/util/0.log" Jul 17 08:07:39.478722 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:39.478702 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066lbr_1448f170-548f-4e5d-a909-d1c97039252a/pull/0.log" Jul 17 08:07:39.494824 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:39.494806 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066lbr_1448f170-548f-4e5d-a909-d1c97039252a/extract/0.log" Jul 17 08:07:39.619728 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:39.619682 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t49tn_84eeb75f-4409-4897-b022-e51ce2d628bf/pull/0.log" Jul 17 08:07:39.626981 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:39.626964 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t49tn_84eeb75f-4409-4897-b022-e51ce2d628bf/extract/0.log" Jul 17 08:07:39.633842 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:39.633826 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t49tn_84eeb75f-4409-4897-b022-e51ce2d628bf/util/0.log" Jul 17 08:07:39.768659 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:39.768628 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzvd9_1835dd21-d9a4-439e-b224-12ddab9997ed/util/0.log" Jul 17 08:07:39.775890 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:39.775865 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzvd9_1835dd21-d9a4-439e-b224-12ddab9997ed/pull/0.log" Jul 17 08:07:39.782087 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:39.782067 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzvd9_1835dd21-d9a4-439e-b224-12ddab9997ed/extract/0.log" Jul 17 08:07:40.067165 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:40.067087 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-nt9qn_1789e5ec-2d38-4951-90b9-62c28eea5c43/manager/0.log" Jul 17 08:07:40.195504 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:40.195475 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-djvpx_c60a3dfd-f974-44b5-b128-6bfe009dd784/manager/0.log" Jul 17 08:07:40.317552 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:40.317470 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-hmtld_be85768d-f7f1-4da5-b8e8-7a58e8c8f9bc/kuadrant-console-plugin/0.log" Jul 17 08:07:40.445523 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:40.445497 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-n5x8d_0f3efd89-40d2-41bb-b3ba-6dafc021373b/registry-server/0.log" Jul 17 08:07:40.849385 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:40.849342 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-bqvbq_b26606a2-fa29-424a-bc05-f350e61cc291/manager/0.log" Jul 17 08:07:41.239952 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:41.239923 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4bnftr_0bef2c56-a999-46b3-96a2-8ef76f96052b/istio-proxy/0.log" Jul 17 08:07:41.614738 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:41.614711 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5f788c4b68-tl59q_89768ae7-e3bf-420a-9c1b-586009b2bff3/kube-auth-proxy/0.log" Jul 17 08:07:41.761974 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:41.761945 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-zd7fg_c7852377-14b6-4b77-98d3-bc5546fdebe5/istio-proxy/1.log" Jul 17 08:07:41.888350 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:41.888206 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-gk6w2_43af3320-dce1-439c-b915-2db00fe459ea/payload-pre-processing/0.log" Jul 17 08:07:42.157405 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:42.157329 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6784546c7b-l47vf_254d445f-a7c6-49e6-a9a6-51b738ccdab2/router/0.log" Jul 17 08:07:42.537403 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:42.537332 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-hxlmg_de34f89f-7f80-4cd0-8842-65634e430cbd/main/0.log" Jul 17 08:07:43.052420 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:43.052377 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-bc6gv_6b8af6fd-3d6c-48d6-9dde-bac6ce028c56/main/0.log" Jul 17 08:07:49.592209 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:49.592168 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-p7q6q/must-gather-bqj9c"] Jul 17 08:07:49.592604 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:49.592568 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9a11756-b5ff-4f48-bc78-ed0d34e498f1" containerName="istio-proxy" Jul 17 08:07:49.592604 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:49.592580 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9a11756-b5ff-4f48-bc78-ed0d34e498f1" containerName="istio-proxy" Jul 17 08:07:49.592604 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:49.592589 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9a11756-b5ff-4f48-bc78-ed0d34e498f1" containerName="istio-proxy" Jul 17 08:07:49.592604 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:49.592594 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9a11756-b5ff-4f48-bc78-ed0d34e498f1" containerName="istio-proxy" Jul 17 08:07:49.592604 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:49.592604 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ef884f3-a294-4171-87a5-7aa8e548fc04" containerName="istio-proxy" Jul 17 08:07:49.592762 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:49.592610 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ef884f3-a294-4171-87a5-7aa8e548fc04" containerName="istio-proxy" Jul 17 08:07:49.592762 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:49.592622 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99d407bd-1ebc-4dcf-91c3-2d728e6fe729" containerName="test-curl-7284-140537757711360" Jul 17 08:07:49.592762 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:49.592628 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="99d407bd-1ebc-4dcf-91c3-2d728e6fe729" containerName="test-curl-7284-140537757711360" Jul 17 08:07:49.592762 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:49.592681 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ef884f3-a294-4171-87a5-7aa8e548fc04" containerName="istio-proxy" Jul 17 08:07:49.592762 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:49.592691 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ef884f3-a294-4171-87a5-7aa8e548fc04" containerName="istio-proxy" Jul 17 08:07:49.592762 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:49.592697 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9a11756-b5ff-4f48-bc78-ed0d34e498f1" containerName="istio-proxy" Jul 17 08:07:49.592762 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:49.592703 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9a11756-b5ff-4f48-bc78-ed0d34e498f1" containerName="istio-proxy" Jul 17 08:07:49.592762 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:49.592708 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="99d407bd-1ebc-4dcf-91c3-2d728e6fe729" containerName="test-curl-7284-140537757711360" Jul 17 08:07:49.593010 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:49.592777 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ef884f3-a294-4171-87a5-7aa8e548fc04" containerName="istio-proxy" Jul 17 08:07:49.593010 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:49.592784 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ef884f3-a294-4171-87a5-7aa8e548fc04" containerName="istio-proxy" Jul 17 08:07:49.595849 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:49.595834 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-p7q6q/must-gather-bqj9c" Jul 17 08:07:49.598110 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:49.598089 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-p7q6q\"/\"kube-root-ca.crt\"" Jul 17 08:07:49.598304 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:49.598273 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-p7q6q\"/\"openshift-service-ca.crt\"" Jul 17 08:07:49.604273 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:49.604253 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-p7q6q/must-gather-bqj9c"] Jul 17 08:07:49.714710 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:49.714676 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/33ba6569-0fcf-4833-acc9-6f90ff246a0c-must-gather-output\") pod \"must-gather-bqj9c\" (UID: \"33ba6569-0fcf-4833-acc9-6f90ff246a0c\") " pod="openshift-must-gather-p7q6q/must-gather-bqj9c" Jul 17 08:07:49.714892 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:49.714730 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dnplw\" (UniqueName: \"kubernetes.io/projected/33ba6569-0fcf-4833-acc9-6f90ff246a0c-kube-api-access-dnplw\") pod \"must-gather-bqj9c\" (UID: \"33ba6569-0fcf-4833-acc9-6f90ff246a0c\") " pod="openshift-must-gather-p7q6q/must-gather-bqj9c" Jul 17 08:07:49.815456 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:49.815421 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/33ba6569-0fcf-4833-acc9-6f90ff246a0c-must-gather-output\") pod \"must-gather-bqj9c\" (UID: \"33ba6569-0fcf-4833-acc9-6f90ff246a0c\") " pod="openshift-must-gather-p7q6q/must-gather-bqj9c" Jul 17 08:07:49.815636 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:49.815478 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnplw\" (UniqueName: \"kubernetes.io/projected/33ba6569-0fcf-4833-acc9-6f90ff246a0c-kube-api-access-dnplw\") pod \"must-gather-bqj9c\" (UID: \"33ba6569-0fcf-4833-acc9-6f90ff246a0c\") " pod="openshift-must-gather-p7q6q/must-gather-bqj9c" Jul 17 08:07:49.815752 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:49.815732 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/33ba6569-0fcf-4833-acc9-6f90ff246a0c-must-gather-output\") pod \"must-gather-bqj9c\" (UID: \"33ba6569-0fcf-4833-acc9-6f90ff246a0c\") " pod="openshift-must-gather-p7q6q/must-gather-bqj9c" Jul 17 08:07:49.824782 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:49.824753 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dnplw\" (UniqueName: \"kubernetes.io/projected/33ba6569-0fcf-4833-acc9-6f90ff246a0c-kube-api-access-dnplw\") pod \"must-gather-bqj9c\" (UID: \"33ba6569-0fcf-4833-acc9-6f90ff246a0c\") " pod="openshift-must-gather-p7q6q/must-gather-bqj9c" Jul 17 08:07:49.906824 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:49.906795 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-p7q6q/must-gather-bqj9c" Jul 17 08:07:50.044117 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:50.044085 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-p7q6q/must-gather-bqj9c"] Jul 17 08:07:50.045003 ip-10-0-138-233 kubenswrapper[2568]: W0717 08:07:50.044975 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod33ba6569_0fcf_4833_acc9_6f90ff246a0c.slice/crio-ec7276b6084c1e9b9aa6227a590435154415d1efc887e476b0f2d5f2f50837e1 WatchSource:0}: Error finding container ec7276b6084c1e9b9aa6227a590435154415d1efc887e476b0f2d5f2f50837e1: Status 404 returned error can't find the container with id ec7276b6084c1e9b9aa6227a590435154415d1efc887e476b0f2d5f2f50837e1 Jul 17 08:07:50.592453 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:50.592419 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-p7q6q/must-gather-bqj9c" event={"ID":"33ba6569-0fcf-4833-acc9-6f90ff246a0c","Type":"ContainerStarted","Data":"ec7276b6084c1e9b9aa6227a590435154415d1efc887e476b0f2d5f2f50837e1"} Jul 17 08:07:51.602612 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:51.602569 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-p7q6q/must-gather-bqj9c" event={"ID":"33ba6569-0fcf-4833-acc9-6f90ff246a0c","Type":"ContainerStarted","Data":"da5147e4675d8cbd42d016de801e18d6f3c3ea1bfbc7b74796c7b56f6579ec5f"} Jul 17 08:07:51.603123 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:51.602630 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-p7q6q/must-gather-bqj9c" event={"ID":"33ba6569-0fcf-4833-acc9-6f90ff246a0c","Type":"ContainerStarted","Data":"958da7d52a060d331fedf49a494c6aec681a276c77068b6ef7455bde68e43dd4"} Jul 17 08:07:51.630512 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:51.624053 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-p7q6q/must-gather-bqj9c" podStartSLOduration=1.9323063409999999 podStartE2EDuration="2.624035062s" totalImagesPullingTime="691.728721ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 08:07:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 08:07:51.621557358 +0000 UTC m=+3376.106246334" watchObservedRunningTime="2026-07-17 08:07:51.624035062 +0000 UTC m=+3376.108724037" Jul 17 08:07:52.399927 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:52.399895 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-4x4z6_2639225a-fbee-4ebd-b36f-7ad47f365e79/global-pull-secret-syncer/0.log" Jul 17 08:07:52.629148 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:52.629113 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-zwtfj_98ab979e-fe39-451d-b8d4-0098ea50c37f/konnectivity-agent/0.log" Jul 17 08:07:52.700763 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:52.700674 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-233.ec2.internal_66a5663e408f43e5d633dcf4f079b8db/haproxy/0.log" Jul 17 08:07:57.146915 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:57.146887 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599rfrw_eb41a9f6-2e49-48dc-aee9-737d3b981c3e/extract/0.log" Jul 17 08:07:57.172100 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:57.172050 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599rfrw_eb41a9f6-2e49-48dc-aee9-737d3b981c3e/util/0.log" Jul 17 08:07:57.200937 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:57.200909 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599rfrw_eb41a9f6-2e49-48dc-aee9-737d3b981c3e/pull/0.log" Jul 17 08:07:57.231502 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:57.231473 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066lbr_1448f170-548f-4e5d-a909-d1c97039252a/extract/0.log" Jul 17 08:07:57.277936 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:57.277907 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066lbr_1448f170-548f-4e5d-a909-d1c97039252a/util/0.log" Jul 17 08:07:57.302761 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:57.302717 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066lbr_1448f170-548f-4e5d-a909-d1c97039252a/pull/0.log" Jul 17 08:07:57.344701 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:57.344673 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t49tn_84eeb75f-4409-4897-b022-e51ce2d628bf/extract/0.log" Jul 17 08:07:57.375097 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:57.375071 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t49tn_84eeb75f-4409-4897-b022-e51ce2d628bf/util/0.log" Jul 17 08:07:57.403884 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:57.403757 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t49tn_84eeb75f-4409-4897-b022-e51ce2d628bf/pull/0.log" Jul 17 08:07:57.436600 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:57.436481 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzvd9_1835dd21-d9a4-439e-b224-12ddab9997ed/extract/0.log" Jul 17 08:07:57.465643 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:57.465618 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzvd9_1835dd21-d9a4-439e-b224-12ddab9997ed/util/0.log" Jul 17 08:07:57.491138 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:57.491083 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzvd9_1835dd21-d9a4-439e-b224-12ddab9997ed/pull/0.log" Jul 17 08:07:58.111342 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:58.111308 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-nt9qn_1789e5ec-2d38-4951-90b9-62c28eea5c43/manager/0.log" Jul 17 08:07:58.140080 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:58.140047 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-djvpx_c60a3dfd-f974-44b5-b128-6bfe009dd784/manager/0.log" Jul 17 08:07:58.168185 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:58.168150 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-hmtld_be85768d-f7f1-4da5-b8e8-7a58e8c8f9bc/kuadrant-console-plugin/0.log" Jul 17 08:07:58.223924 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:58.223894 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-n5x8d_0f3efd89-40d2-41bb-b3ba-6dafc021373b/registry-server/0.log" Jul 17 08:07:58.485351 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:07:58.485217 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-bqvbq_b26606a2-fa29-424a-bc05-f350e61cc291/manager/0.log" Jul 17 08:08:00.329295 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:00.329222 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-xr6pg_2ce00a54-15e9-42b7-b3a8-9e58d1101558/cluster-monitoring-operator/1.log" Jul 17 08:08:00.427937 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:00.427898 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-xr6pg_2ce00a54-15e9-42b7-b3a8-9e58d1101558/cluster-monitoring-operator/0.log" Jul 17 08:08:00.636986 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:00.636959 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-g4gmw_455b815f-ec90-4a7e-bef3-7b4e202791cb/node-exporter/0.log" Jul 17 08:08:00.663149 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:00.663120 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-g4gmw_455b815f-ec90-4a7e-bef3-7b4e202791cb/kube-rbac-proxy/0.log" Jul 17 08:08:00.691967 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:00.691939 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-g4gmw_455b815f-ec90-4a7e-bef3-7b4e202791cb/init-textfile/0.log" Jul 17 08:08:02.190842 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:02.190793 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-p7q6q/perf-node-gather-daemonset-mdjfh"] Jul 17 08:08:02.198825 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:02.198791 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-p7q6q/perf-node-gather-daemonset-mdjfh" Jul 17 08:08:02.203155 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:02.203124 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-p7q6q\"/\"default-dockercfg-9bq8c\"" Jul 17 08:08:02.207008 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:02.206976 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-p7q6q/perf-node-gather-daemonset-mdjfh"] Jul 17 08:08:02.244399 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:02.244359 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f5206fa7-3da5-4579-87dc-4a54eaeb10c8-podres\") pod \"perf-node-gather-daemonset-mdjfh\" (UID: \"f5206fa7-3da5-4579-87dc-4a54eaeb10c8\") " pod="openshift-must-gather-p7q6q/perf-node-gather-daemonset-mdjfh" Jul 17 08:08:02.244595 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:02.244423 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f5206fa7-3da5-4579-87dc-4a54eaeb10c8-lib-modules\") pod \"perf-node-gather-daemonset-mdjfh\" (UID: \"f5206fa7-3da5-4579-87dc-4a54eaeb10c8\") " pod="openshift-must-gather-p7q6q/perf-node-gather-daemonset-mdjfh" Jul 17 08:08:02.244595 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:02.244563 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxjlj\" (UniqueName: \"kubernetes.io/projected/f5206fa7-3da5-4579-87dc-4a54eaeb10c8-kube-api-access-dxjlj\") pod \"perf-node-gather-daemonset-mdjfh\" (UID: \"f5206fa7-3da5-4579-87dc-4a54eaeb10c8\") " pod="openshift-must-gather-p7q6q/perf-node-gather-daemonset-mdjfh" Jul 17 08:08:02.244698 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:02.244628 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f5206fa7-3da5-4579-87dc-4a54eaeb10c8-sys\") pod \"perf-node-gather-daemonset-mdjfh\" (UID: \"f5206fa7-3da5-4579-87dc-4a54eaeb10c8\") " pod="openshift-must-gather-p7q6q/perf-node-gather-daemonset-mdjfh" Jul 17 08:08:02.244698 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:02.244683 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f5206fa7-3da5-4579-87dc-4a54eaeb10c8-proc\") pod \"perf-node-gather-daemonset-mdjfh\" (UID: \"f5206fa7-3da5-4579-87dc-4a54eaeb10c8\") " pod="openshift-must-gather-p7q6q/perf-node-gather-daemonset-mdjfh" Jul 17 08:08:02.345415 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:02.345376 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f5206fa7-3da5-4579-87dc-4a54eaeb10c8-podres\") pod \"perf-node-gather-daemonset-mdjfh\" (UID: \"f5206fa7-3da5-4579-87dc-4a54eaeb10c8\") " pod="openshift-must-gather-p7q6q/perf-node-gather-daemonset-mdjfh" Jul 17 08:08:02.345680 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:02.345662 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f5206fa7-3da5-4579-87dc-4a54eaeb10c8-lib-modules\") pod \"perf-node-gather-daemonset-mdjfh\" (UID: \"f5206fa7-3da5-4579-87dc-4a54eaeb10c8\") " pod="openshift-must-gather-p7q6q/perf-node-gather-daemonset-mdjfh" Jul 17 08:08:02.345814 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:02.345790 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f5206fa7-3da5-4579-87dc-4a54eaeb10c8-lib-modules\") pod \"perf-node-gather-daemonset-mdjfh\" (UID: \"f5206fa7-3da5-4579-87dc-4a54eaeb10c8\") " pod="openshift-must-gather-p7q6q/perf-node-gather-daemonset-mdjfh" Jul 17 08:08:02.345895 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:02.345684 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f5206fa7-3da5-4579-87dc-4a54eaeb10c8-podres\") pod \"perf-node-gather-daemonset-mdjfh\" (UID: \"f5206fa7-3da5-4579-87dc-4a54eaeb10c8\") " pod="openshift-must-gather-p7q6q/perf-node-gather-daemonset-mdjfh" Jul 17 08:08:02.346071 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:02.346051 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxjlj\" (UniqueName: \"kubernetes.io/projected/f5206fa7-3da5-4579-87dc-4a54eaeb10c8-kube-api-access-dxjlj\") pod \"perf-node-gather-daemonset-mdjfh\" (UID: \"f5206fa7-3da5-4579-87dc-4a54eaeb10c8\") " pod="openshift-must-gather-p7q6q/perf-node-gather-daemonset-mdjfh" Jul 17 08:08:02.346251 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:02.346237 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f5206fa7-3da5-4579-87dc-4a54eaeb10c8-sys\") pod \"perf-node-gather-daemonset-mdjfh\" (UID: \"f5206fa7-3da5-4579-87dc-4a54eaeb10c8\") " pod="openshift-must-gather-p7q6q/perf-node-gather-daemonset-mdjfh" Jul 17 08:08:02.346405 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:02.346391 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f5206fa7-3da5-4579-87dc-4a54eaeb10c8-proc\") pod \"perf-node-gather-daemonset-mdjfh\" (UID: \"f5206fa7-3da5-4579-87dc-4a54eaeb10c8\") " pod="openshift-must-gather-p7q6q/perf-node-gather-daemonset-mdjfh" Jul 17 08:08:02.346511 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:02.346492 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f5206fa7-3da5-4579-87dc-4a54eaeb10c8-proc\") pod \"perf-node-gather-daemonset-mdjfh\" (UID: \"f5206fa7-3da5-4579-87dc-4a54eaeb10c8\") " pod="openshift-must-gather-p7q6q/perf-node-gather-daemonset-mdjfh" Jul 17 08:08:02.346628 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:02.346451 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f5206fa7-3da5-4579-87dc-4a54eaeb10c8-sys\") pod \"perf-node-gather-daemonset-mdjfh\" (UID: \"f5206fa7-3da5-4579-87dc-4a54eaeb10c8\") " pod="openshift-must-gather-p7q6q/perf-node-gather-daemonset-mdjfh" Jul 17 08:08:02.359070 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:02.357660 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxjlj\" (UniqueName: \"kubernetes.io/projected/f5206fa7-3da5-4579-87dc-4a54eaeb10c8-kube-api-access-dxjlj\") pod \"perf-node-gather-daemonset-mdjfh\" (UID: \"f5206fa7-3da5-4579-87dc-4a54eaeb10c8\") " pod="openshift-must-gather-p7q6q/perf-node-gather-daemonset-mdjfh" Jul 17 08:08:02.513583 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:02.513485 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-p7q6q/perf-node-gather-daemonset-mdjfh" Jul 17 08:08:02.721071 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:02.718621 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-p7q6q/perf-node-gather-daemonset-mdjfh"] Jul 17 08:08:03.375247 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:03.375207 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-xpxv5_27538d1e-992b-4651-a2d2-7a7ecf442806/console-operator/1.log" Jul 17 08:08:03.382000 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:03.381972 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-xpxv5_27538d1e-992b-4651-a2d2-7a7ecf442806/console-operator/2.log" Jul 17 08:08:03.709511 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:03.709416 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-p7q6q/perf-node-gather-daemonset-mdjfh" event={"ID":"f5206fa7-3da5-4579-87dc-4a54eaeb10c8","Type":"ContainerStarted","Data":"365ceb7f38feda64301fc652373d07f2edc92d88dbe744948494bd357fc2a5fe"} Jul 17 08:08:03.709511 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:03.709486 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-p7q6q/perf-node-gather-daemonset-mdjfh" event={"ID":"f5206fa7-3da5-4579-87dc-4a54eaeb10c8","Type":"ContainerStarted","Data":"d8aecc2039580950d9e50ce30719a7a0a988af6a1118c313a70a6920378656a7"} Jul 17 08:08:03.710375 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:03.710347 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-p7q6q/perf-node-gather-daemonset-mdjfh" Jul 17 08:08:03.728323 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:03.728244 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-p7q6q/perf-node-gather-daemonset-mdjfh" podStartSLOduration=1.7282245459999999 podStartE2EDuration="1.728224546s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 08:08:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 08:08:03.727735824 +0000 UTC m=+3388.212424799" watchObservedRunningTime="2026-07-17 08:08:03.728224546 +0000 UTC m=+3388.212913521" Jul 17 08:08:04.530690 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:04.530657 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-xgtz8_6462d06e-f55f-44cc-9913-ef8120a6f4cc/volume-data-source-validator/0.log" Jul 17 08:08:05.668466 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:05.668435 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vc255_b1d8b7b7-173b-4e59-acce-7fbe2b1f5465/dns/0.log" Jul 17 08:08:05.691908 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:05.691883 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vc255_b1d8b7b7-173b-4e59-acce-7fbe2b1f5465/kube-rbac-proxy/0.log" Jul 17 08:08:05.776409 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:05.776381 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-t6g42_6a12d3cc-3a1a-4241-92b6-7d6198bc33c3/dns-node-resolver/0.log" Jul 17 08:08:06.386740 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:06.386710 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-cw57w_3bc6540c-1f26-4a5e-9f66-e3480272699d/node-ca/0.log" Jul 17 08:08:07.407629 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:07.407594 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4bnftr_0bef2c56-a999-46b3-96a2-8ef76f96052b/istio-proxy/0.log" Jul 17 08:08:07.575482 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:07.575452 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5f788c4b68-tl59q_89768ae7-e3bf-420a-9c1b-586009b2bff3/kube-auth-proxy/0.log" Jul 17 08:08:07.605785 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:07.605749 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-zd7fg_c7852377-14b6-4b77-98d3-bc5546fdebe5/istio-proxy/0.log" Jul 17 08:08:07.643267 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:07.643216 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-zd7fg_c7852377-14b6-4b77-98d3-bc5546fdebe5/istio-proxy/1.log" Jul 17 08:08:07.676737 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:07.676655 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-gk6w2_43af3320-dce1-439c-b915-2db00fe459ea/payload-pre-processing/0.log" Jul 17 08:08:07.745412 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:07.745380 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6784546c7b-l47vf_254d445f-a7c6-49e6-a9a6-51b738ccdab2/router/0.log" Jul 17 08:08:08.442453 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:08.442422 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2n5zd_2df1cbe7-b30d-450f-b2e8-c3f4e10d00d5/serve-healthcheck-canary/0.log" Jul 17 08:08:09.020551 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:09.020513 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-jv9pk_2a3b3827-cd5d-4292-8ce8-4e1c17c4c1e4/insights-operator/0.log" Jul 17 08:08:09.022579 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:09.022555 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-jv9pk_2a3b3827-cd5d-4292-8ce8-4e1c17c4c1e4/insights-operator/1.log" Jul 17 08:08:09.206295 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:09.206256 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-khqt9_9a4455d4-a377-4848-a454-54f3710b4a02/kube-rbac-proxy/0.log" Jul 17 08:08:09.231141 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:09.231112 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-khqt9_9a4455d4-a377-4848-a454-54f3710b4a02/exporter/0.log" Jul 17 08:08:09.256362 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:09.256331 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-khqt9_9a4455d4-a377-4848-a454-54f3710b4a02/extractor/0.log" Jul 17 08:08:10.733015 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:10.732987 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-p7q6q/perf-node-gather-daemonset-mdjfh" Jul 17 08:08:11.450419 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:11.450386 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-nznpn_563e3f68-d742-4e2b-8cab-03385da4780c/manager/0.log" Jul 17 08:08:11.629613 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:11.629578 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-kskbn_92188035-d0d9-4c04-b496-0d53544cde37/manager/0.log" Jul 17 08:08:11.774191 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:11.774020 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-579d999db4-94vrg_222416ad-c37f-4cc0-946f-7c11384a9f91/manager/0.log" Jul 17 08:08:11.800452 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:11.800408 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-smgzc_14f64e3c-e222-4720-b723-358eeed0c402/server/0.log" Jul 17 08:08:11.882739 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:11.882709 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-7kqk8_8dc175df-3be1-4f44-aaed-fe6ee8e60771/manager/0.log" Jul 17 08:08:11.977470 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:11.977438 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59ffcf4dc8-56dhq_6a4b5d36-d807-4d62-8aaf-6432767c269c/manager/0.log" Jul 17 08:08:12.000997 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:12.000971 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59ffcf4dc8-56dhq_6a4b5d36-d807-4d62-8aaf-6432767c269c/copy-manifests/0.log" Jul 17 08:08:12.310361 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:12.310303 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4bnftr_0bef2c56-a999-46b3-96a2-8ef76f96052b/istio-proxy/0.log" Jul 17 08:08:12.449827 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:12.449800 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5f788c4b68-tl59q_89768ae7-e3bf-420a-9c1b-586009b2bff3/kube-auth-proxy/0.log" Jul 17 08:08:12.478345 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:12.478310 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-zd7fg_c7852377-14b6-4b77-98d3-bc5546fdebe5/istio-proxy/0.log" Jul 17 08:08:12.526694 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:12.526611 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-zd7fg_c7852377-14b6-4b77-98d3-bc5546fdebe5/istio-proxy/1.log" Jul 17 08:08:12.549338 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:12.549310 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-gk6w2_43af3320-dce1-439c-b915-2db00fe459ea/payload-pre-processing/0.log" Jul 17 08:08:12.591990 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:12.591961 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6784546c7b-l47vf_254d445f-a7c6-49e6-a9a6-51b738ccdab2/router/0.log" Jul 17 08:08:13.914887 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:13.914858 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4bnftr_0bef2c56-a999-46b3-96a2-8ef76f96052b/istio-proxy/0.log" Jul 17 08:08:14.056920 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:14.056893 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5f788c4b68-tl59q_89768ae7-e3bf-420a-9c1b-586009b2bff3/kube-auth-proxy/0.log" Jul 17 08:08:14.081965 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:14.081546 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-zd7fg_c7852377-14b6-4b77-98d3-bc5546fdebe5/istio-proxy/0.log" Jul 17 08:08:14.123051 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:14.123019 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-zd7fg_c7852377-14b6-4b77-98d3-bc5546fdebe5/istio-proxy/1.log" Jul 17 08:08:14.145681 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:14.145655 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-gk6w2_43af3320-dce1-439c-b915-2db00fe459ea/payload-pre-processing/0.log" Jul 17 08:08:14.188369 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:14.188301 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6784546c7b-l47vf_254d445f-a7c6-49e6-a9a6-51b738ccdab2/router/0.log" Jul 17 08:08:18.968625 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:18.968588 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-44ftt_4c8e8c69-d8f8-4285-bed5-b527c4d915f8/kube-storage-version-migrator-operator/1.log" Jul 17 08:08:18.969720 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:18.969697 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-44ftt_4c8e8c69-d8f8-4285-bed5-b527c4d915f8/kube-storage-version-migrator-operator/0.log" Jul 17 08:08:20.591073 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:20.590990 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-z2q9x_4fbcebdd-a629-4d9a-9ba1-39e350bfa786/kube-multus-additional-cni-plugins/0.log" Jul 17 08:08:20.615944 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:20.615916 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-z2q9x_4fbcebdd-a629-4d9a-9ba1-39e350bfa786/egress-router-binary-copy/0.log" Jul 17 08:08:20.645249 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:20.645211 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-z2q9x_4fbcebdd-a629-4d9a-9ba1-39e350bfa786/cni-plugins/0.log" Jul 17 08:08:20.672940 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:20.672909 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-z2q9x_4fbcebdd-a629-4d9a-9ba1-39e350bfa786/bond-cni-plugin/0.log" Jul 17 08:08:20.697205 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:20.697177 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-z2q9x_4fbcebdd-a629-4d9a-9ba1-39e350bfa786/routeoverride-cni/0.log" Jul 17 08:08:20.720807 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:20.720780 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-z2q9x_4fbcebdd-a629-4d9a-9ba1-39e350bfa786/whereabouts-cni-bincopy/0.log" Jul 17 08:08:20.744803 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:20.744748 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-z2q9x_4fbcebdd-a629-4d9a-9ba1-39e350bfa786/whereabouts-cni/0.log" Jul 17 08:08:20.790984 ip-10-0-138-233 kubenswrapper[2568]: I0717 08:08:20.790929 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-nfjfq_19b25751-5376-4627-9965-d850af34a46d/kube-multus/0.log"